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ON THE SAMPLING ERRORS OF FACTOR LOADINGS 


QUINN MCNEMAR 
STANFORD UNIVERSITY 


The results of three empirical studies on the sampling fluctua- 
tion of centroid factor loadings are reported. The first study is 
based on data which happened to be available on 8 variables for 700 
cases and which were factored to three factors for subsamples. The 
second study is based on fictitious data for 2500 cases which provided 
separate analyses on 25 samples for each of three situations: 5 vari- 
ables, one factor; 5 variables, two factors; and 6 variables, three fac- 
tors. The third study, based on real data for 9 variables and 7000 
cases, involves separate factorization for 35 samples of 200 cases. 
The three studies agree in showing that the sampling behavior of first 
centroid factor loadings is much like that of correlation coefficients, 
whereas the sampling fluctuations for loadings beyond the first are 
disturbingly large. 


Those who have been instrumental in the recent development of 
factorial methods to a high degree of mathematical nicety have been 
so casual in their treatment of sampling that many who are apply- 
ing the new methods have failed to realize that the data for analysis 
are basically statistical in nature, and therefore subject to sampling 
errors. Sampling can affect the results of factor analysis via the 
stability of loadings, the number of factors to be extracted, and the 
location of the rotated factorial axes. This paper will be concerned 
primarily with the first of these, but the results will of course have 
implications for the third. The problem of sampling is that of de- 
termining not only the nature and extent of the distribution of suc- 
cessive sample values but also the presence of bias, i.e., whether the 
sampling distributions center about the universe or population para- 
meters. It is not easy, however, to hypothecate a universe factor 
loading and conceive of sampling fluctuations therefrom, nor is it 
easy to imagine the sampling behavior of axes, either centroid or 
rotated. 

Thurstone’s assertion that simple structure will not arise as a 
result of chance sampling, though acceptable on theoretical grounds, 
is not sufficient in actual situations wherein the structure may be so 
lacking in conciseness as to involve a great deal of subjective arbitrari- 
ness. The struggle to put psychological meaning into the results of 
factor analyses has been greatly befuddled by the presence of chance 
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errors, the nature and magnitude of which are unknown. Perhaps 
many imaginative investigators have succeeded in assigning seeming- 
ly rational meaning to that which has resulted largely from chance, 
and, therefore, some of the traits supposedly isolated may be only 
imaginative. Aside from the need to know something of the magnitude 
of sampling errors in studies directed primarily at the isolation of 
traits or abilities, there is even greater need for this information in 
connection with such problems as the relationship of organization to 
age, the effect of changes experimentally produced (e.g., practice), 
and the invariability of the factorial description of tests which are 
moved from one battery to another when the two batteries have been 
administered to two different groups of subjects. 

Ideally the size of sampling errors in factorial analysis should be 
determined analytically, but the non-analytic nature of certain steps 
in the Thurstone method, the method which at present seems most de- 
fensible psychologically, precludes the analytic approach. It would 
seem appropriate to attack the problem empirically. The writer’s first 
approach was by way of data which just happened to be available. A 
group of 700 cases with scores on 8 variables was broken down into 
14 subsamples of 50 each and 7 subsamples of 100 each. The product 
moment correlational matrices were factorized to three factors. The 
results can be summarized briefly by saying that the first centroid 
loadings tended to fluctuate to about the same extent as product mo- 
ment correlation coefficients and that the second and third centroid 
loadings showed greater fluctuation — about twice that of the first 
factor loadings. 


Fictitious Data 


Our second approach consisted in setting up “scores” on 7 vari- 
ables for 2500 cases in such a manner that 5 variables with one factor, 
5 variables with two factors, and 6 variables with 3 factors could be 
chosen for analysis. The scores were obtained by counting the number 
of odd digits in columns of Barlow’s Tables (Tippett’s random num- 
bers are not extensive enough for our purpose), one column corres- 
ponding to an individual, and 15 columns being utilized per page be- 
ginning with page 22. The variables were defined in terms of rows, 
and the numbers of odds were counted and summed so as to provide 
common elements between some variables and elements specific to 
each variable. The set-up in terms of elements is presented in Table 1. 

It was assumed that a digit’s being odd (or even) is a chance 
matter, but of course this isn’t true — some columns were not used as 
cases because of obvious runs. Now this lack of randomness can be 
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expected to operate so as to interfere with the obtaining of 2500 cases 
with scores, and consequent intercorrelations, of exactly the factorial 
composition indicated in Table 1. This would invalidate these data for 
use in setting up a criterion for the number of factors, but so far as 
the present sampling problem is concerned, we need not worry if the 
2500 cases do not yield intercorrelations sufficiently in accord with the 
theoretically expected correlations. As a matter of fact, some of the 
empirical 7’s based on all 2500 cases are significantly different from 
the expected values. These discrepancies, indicating definitely that 
the numbers in Barlow do not exhibit randomness as regards being 
odd or even, do not nullify the data for our purpose. In other words, 
when we have determined scores for 2500 cases on the 7 variables, we 
can safely ignore this inadequacy in setting up the scores, providing 
our method for breaking the 2500 cases down into subsamples is 
random and independent of the scheme utilized in setting up the 
scores. 

The 2500 cases were randomly (we believe) divided into 25 
groups of 100 each by the following procedure. The cases were serially 
numbered from 1 to 2500, sorted into 10 piles of 250 each on the basis 
of the digit’s column of the serial number on the Hollerith cards, then 
each pile of 250 was sorted on the basis of the ten’s column into 10 
groups of 25 each, and finally four such small groups were combined 
for samples of 100 each. The writer sees no reason why this should 
not lead to random samples. 

The intercorrelations for the successive 25 samples. and for the 
total group were computed as tetrachoric 7’s with cuts near the me- 
dians. The use of tetrachorics will of course lead to larger sampling 
errors than would obtain if product moment r’s were involved. The 
7 variables had been arranged so as to permit the following series of 
factor analyses: five variables (1, 2, 3, 4, and 7) with one factor; five 
variables (1, 2, 4, 5, and 6) with two factors; six variables (1, 3, 4, 
5, 6, and 7) with three factors. For each of these series, 26 analyses 
were carried out, thus permitting the 25 sets of subsample loadings to 
be compared with the loadings obtained by analyzing the total group. 
No corrections for communality estimates were utilized. (For one 
series, the several analyses were carried through on the basis of second 
or revised estimates, but this did not lead to an appreciable change in 
the results.) 

A word should be inserted about sign changing or test reflection. 
The procedure followed was a combination of the number of negative 
signs and the ultimate criterion that column sums should not be nega- 
tive. As regards the proper sign to be given to the loadings, an 
attempt was made to have the signs in each column in the sample 
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factorial matrices be in maximum agreement with the signs in the 
corresponding matrices based on the total 2500. 

The results of the several series are reported in Tables 2, 3, and 
4. The entries in these tables are nearly self-explanatory: the rows 
marked 2500 give the loadings which resulted from factorizing the 
correlations based on the total group; the next row gives the mean 
loading for the 25 samples; and S.D. is the sigma of the sampling dis- 
tribution of loadings. These §.D.’s presumably approximate the 
standard errors of the centroid factor loadings. 

It will be noted from these tables that the first factor loadings 
exhibit sampling fluctuations of about the same order as the chance 
errors of tetrachoric correlations of magnitudes corresponding to the 
factor loadings and based on N’s of 100. It will also be noted that 
there is some tendency for the size of these sampling errors to be de- 
pendent upon the magnitude of the “universe” loading. This parallels 
the known sampling behavior of correlation coefficients. The second 
and third factor loadings show rather marked fluctuations. If these 
data are at all valid, it would seem that loadings beyond the first are 
rather unstable from the sampling viewpoint. 

The results in Tables 2, 3, and 4 are in terms of centroid loadings. 
One wonders what would happen in case the axes were rotated, but 
certain difficulties arise here—the sampling fluctuations of projections 
on rotated axes would involve taking deviations from axes which 
would themselves be subject to sampling variability plus arbitrariness 
in the rotations. It is here that the fairly close accord between the 
means of 25 sample loadings and those of the “universe” loadings, 
based on all 2500 cases, takes on importance. It would seem that the 
centroid projections do fluctuate about the “universe” centroid load- 
ings. 

Aside from the sampling distribution of loadings, one further 
question may be raised. What is the consequent effect of sampling on 
the test clusters? Or to what extent do the tests tend to show the 
same clustering for a sample as for the “universe” from which the 
sample is drawn? With regard to the five variable, two factor series, 
it will be noted that the set-up for these variables (see Table 1) is 
such that, theoretically, one would expect an axis through tests 1 and 
2 to be orthogonal to an axis through tests 5 and 6, and that test 4 
would fall between these axes. This holds true for the analysis based 
upon all 2500 cases, but when the factor plots for the several 25 
samples were examined it was discovered that five samples yielded 
clusters decidedly different from the “universe” pattern. The remain- 
ing 20 samples show fair agreement, i.e., one would arrive at an 
interpretation therefrom which would not be much at variance with 
that deduced from the “universe.” 
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When a similar examination was made of the factor patterns for 
the six variable, three factor set-up, it was found that in at least 10 of 
the 25 samples there would be questions as to whether the clusters for 
the samples agreed with that for the “universe.” These divergences, 
it should be noted, cannot be rectified by rotations. Further discus- 
sion will be postponed until after the results of a third approach are 
presented. 


College Entrance Board Data* 


The scores for 7000 cases on 9 variables were punched on Hol- 
lerith cards in such a manner as to permit the determination of tetra- 
chorics with near median cuts. The case numbers from the test book- 
lets were also included on the cards, which were serialized by these 
numbers, and then sorted by hand into 35 piles of 200 cards each, the 
1st card to pile 1, the 2nd to pile 2, --- , the 35th to pile 35, the 36th 
to pile 1, etc. The intercorrelations for these 35 random samples and 
for the total 7000 were computed, and the 36 resulting correlational 
matrices were factorized to three factors. Although the magnitudes of 
the residuals indicate rather definitely that at least three factors have 
statistical significance for the “universe,” i.e., the total 7000, one may 
well question the significance of more than two factors for the sam- 
ples of size 200. It is of interest, however, to know how widely the 
third factor loadings vary from sample to sample even though a pos- 
sibly valid criterion for the number of factors would suggest that a 
third factor should not be extracted for these samples of 200 cases. 

It should again be noted that the signs in the columns of each of 
the several factorial matrices were adjusted by changing all the signs, 
if necessary, in a column so as to get the maximum sign agreement 
with the corresponding column of the “universe” factorial matrix. 

The distributions for the factor loadings resulting from the 35 
successive samples will be found in Tables 5, 6, and 7. At the bottom 
of each distribution will be found the “universe” loadings, the means 
for the sampling values, and the sigmas of the distributions. (The 
means and sigmas were computed from distributions with finer group- 
ings). From these tables, there would seem to emerge a number of 
pertinent facts, facts which are consistent with the published and un- 
published results of the less adequate fictitious data. 

In the first place, the means for the first centroid loadings agree 
quite closely with the “universe” loadings, the largest discrepancy be- 
ing .007, whereas the correspondence of the means for second and 
third loadings is not particularly good. 


* The writer is grateful to Professors John M. Stalnaker and Carl C. Brig- 
ham for these data. 
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In the second place, the sampling distributions for the first factor 
loadings show dispersions which are comparable with that expected 
for the sampling distribution of correlation coefficients (tetrachoric) 
based on samples of, in this case, 200. As regards loadings for factors 
beyond the first, it will be noted that the dispersions for the second 
centroid loadings are more than double those for the first, while the 
third factor projections yield distributions which have standard devia- 
ations approximately three times as large as those for the first axis, 

In the third place, there are certain features of the distributions 
which need to be mentioned. There is a noticeable tendency for the 
distributions in Table 5 to be skewed in much the same fashion as 
might be anticipated if the sampling behavior of high factor loadings 
were similar to that for high correlation coefficients. Nearly all the 
distributions for the second and third factors exhibit a marked ten- 
dency towards bimodality. It follows, therefore, that means and sig- 
mas are not sufficient as descriptive measures for these distributions. 
The writer, without applying significance tests, believes that the ap- 
parent bimodality is real, that it is inherent in the centroid method, 
and that it is largely due to the necessity for reflecting tests so as to 
remove the centroid from zero for residual tables. It should be obvi- 
ous that, for any one test, the signs for all 35 sample factor loadings 
could be made to agree by an appropriate arrangement of all the signs 
in the proper column of the several sample factorial matrices and thus 
the bimodality for that test could be eliminated. Such a procedure, 
however, would not lead to as close agreement with the “universe” 
values for all tests considered simultaneously. As already indicated, 
the distributions in Tables 6 and 7 represent the maximum amount of 
agreement between sample and universe values as regards the signs 
of the loadings. Any one test could be brought into closer agreement 
but only at the expense of poorer agreement for some other test or 
tests. 

Aside from these features of the sampling distributions, let us 
consider the effect of sampling on the test clusters or factor patterns. 
An examination of the plots reveals that no less than 15 of the 35 sam- 
ples have yielded clusters which are different from that for the uni- 
verse. This is somewhat worse than that found for the fictitious data 
despite the fact that for the College Entrance Board Data we have a 
larger number of variables and larger samples. This may be due to 
the fact that the fictitious data were so set up that definite significance 
could be attached to the second and third factors, whereas the signifi- 
cance or meaning of factors beyond the first for the Board Data is 
open to question. For the total 7000 cases, tests 2 and 6 (both numer- 
ical) tend to hang together, but aside from this the writer has been 
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unable to make sense of the factor pattern. Perhaps, therefore, in ex- 
amining the patterns for the 35 successive samples, one should pay 
particular attention to the behavior of tests 2 and 6. When this is 
done, it is found that these two tests do tend to hang together from 
sample to sample, but one wonders what significance can be assigned 
to this fact in those sample situations wherein other tests irrationally 
join the numerical “cluster.” 

It might be claimed that no significance can be assigned to the 
second and third factor loadings for the universe from which the sam- 
ples were drawn, and consequently that the results for the samples 
are trifling. This would be equivalent to saying that centroid loadings 
as high as .40 to .60 are trifling even though based on a sample of 
200 (equivalent in sampling effectiveness to about 100 cases when 
product moment 7’s are used). The important point, in the writer’s 
opinion, is that the sampling fluctuation of loadings for factors be- 
yond the first is apparently so great that large loadings may readily 
occur by chance. Incidentally, the writer is not convinced that the re- 
sults of analyses based on relatively small samples have more than 
chance significance merely because an investigator, after an obvious 
struggle, succeeds in assigning supposedly rational meanings to his 


findings. 


Discussion 


The results of these studies on the effect of sampling on factor 
loadings are consistent in showing that first factor loadings behave 
like correlation coefficients, whereas loadings beyond the first not only 
show greater fluctuation but also tend to bimodality. The implications 
of these findings, taken at their face value, would seem quite obvious. 
There are, however, certain limitations to these studies which need 
to be stressed, the most serious of which is the smallness in the num- 
ber of variables. The inclusion of more variables will not necessarily 
iron out the effect of the chance errors of the original correlation co- 
efficients since these sampling errors are not independent, i.e., they 
are correlated. One might expect that the use of additional or revised 
estimates of communalities would result in slightly greater stability 
for the factor loadings. The results herein are limited to Thurstone’s 
centroid method. Whether other factorial methods are subject to simi- 
lar sampling fluctuations remains to be determined. 

There is one point of theoretical interest which rapidly goes be- 
yond the mathematical comprehension of the present writer. In other 
sampling situations it is usually easy to conceive of a universe value 
about which sample values will disperse themselves. In the factorial 
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situation, it is difficult to see just what the universe value or values 
might be. The statistic which varies from sample to sample is the fac- 
tor loading or projection on a centroid axis which itself varies from 
sample to sample. A mere rotation of axes, regardless of how clear- 
cut the structure, will not change the predicament—the location of 
the rotated axes is subject to the chance vagaries of sampling, and 
of course, the test projections thereupon are also affected by sampling. 
In other words, we have no stable reference frame from which to re- 
gard the sampling fluctuations of test projections. Suppose we have n 
variables which for the universe yield a simple structure of exactly r 
primaries. A sample of 200 is drawn randomly. Neither the centroid 
nor the rotated axes can be thought of as necessarily coinciding with 
the axes for the universe, and therefore the deviations of the loadings 
from universe values will not be determinate. This may constitute a 
serious theoretical limitation to our empirical study. 

If the results of these empirical studies are to be taken as even 
approximating the truth, it follows that psychologists might very well 
cease filling the literature with factor analyses based on, say, less than 
100 cases. Furthermore, if the sampling situation in factor analysis 
is anything like that in other branches of statistics, one might very 
well expect that somewhere along the line the concept of degrees of 
freedom will enter in such a manner that the effective size of the sam- 
ple is diminished by the number of variables in the analysis. The 
writer is not anxious specifically to cite any particular study, but he 
does find himself wondering about the sampling effectiveness of a fac- 
tor study of 52 variables based on tetrachoric r’s for 110 cases. The 
use of tetrachorics automatically reduces the effective size of the 
sample to about 55, whence there may be left but 3 degrees of freedom 
or an effective N of only 3. Perhaps this explains why so much diffi- 
culty was encountered when trying to make psychological sense of 
the results. 


TABLE 1 
Number of common and specific elements for fictitious data 
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TABLE 2 


Five variables, one factor 


First factor 


1 2 3 4 7 

2500 83 69 69 81 -76 

Mean 83 69 -70 .80 75 

8.D. 07 12 .10 .09 sit 
TABLE 3 


Five variables, two factors 


First factor 


1 2 4 5 6 
2500 .73 63 86 25 32 
Mean 71 63 85 .29 34 
S.D. 14 14 .09 .18 25 


Second factor 


2500 42 33 .06 —.36 —.37 

Mean 41 36 .09 —.45 —.36 

S.D. 16 17 19 18 21 
TABLE 4 


Six variables, three factors 


First factor 


1 3 4 5 6 7 
2500 -76 61 .86 18 33 -76 
Mean 75 61 85 .22 34 74 
S.D 12 1 .09 14 .23 12 


Second factor 


2500 33 36 —.09 —.44 —.35 22 
Mean 37 36 .00 —.48 —.40 .20 
S.D. 15 20 .20 15 19 .20 


Third factor 


2500 .18 10 .20 14 —.31 —.28 
Mean .10 02 .10 .28 —.26 —.16 
S.D. .24 27 .23 15 12 .26 
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College Entrance Board Data 
Sampling distribution for first factor loadings 
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College Entrance Board Data 
Sampling distribution for second factor loadings 
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TABLE 7 


College Entrance Board Data 
Sampling distribution for third factor loadings 
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TEST RELIABILITY ESTIMATED BY ANALYSIS 
OF VARIANCE 


CYRIL Hoyt 
UNIVERSITY OF MINNESOTA 


A formula for estimating the reliability of a test, based on the 
analysis of variance theory, is developed and illustrated. The data 
needed for the required computation are the number of correct re- 
sponses to each item and the score for each subject. The results 
obtained from this formula are identical with those from one of the 
special cases of the Kuder-Richardson formulation. The relation- 
rar gad new procedure to other approaches to the problem are 
indicated. 


This paper is composed of two parts, the first of which describes 
the procedures followed in computing the estimated reliability of a 
test by analysis of variance while the second part contains the mathe- 
matical derivation of the formulas. 


I 


The coefficient of reliability of a test gives the percentage of the 
obtained variance in the distribution of test scores that may be re- 
garded as true variance, that is, as variance not due to the unreliabil- 
ity of the measuring instrument. Out of Rulon’s *work on a short 


method of estimating the reliability coefficient by means of “split- 
halves” comes the relationship 
oa’ 
r,—1 sf 
where ao,” is the variance of the distribution of differences obtained by 
subtracting a student’s score on the odd items of a test from that on 
the even items, or vice versa, and o;? is the variance of the distribu- 
tion of the students’ scores on the test. From this information, it is 
apparent that o,? is used as a measure of the discrepancy between the 
obtained variance and the true variance, o,.2. If the odd-even split of 
the test happens to be an unlucky one, o,? as thus computed may be an 
underestimate or an overestimate of the discrepancy between the ob- 
tained variance and the true variance. It was this difficulty that led 


* Rulon, Phillip J. A simplified procedure for determining the reliability of 
a test by split-halves. Harvard Educational Review, 1989, 9, 99-108. 
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the writer to seek a better estimate of this discrepancy. Table I shows 
the results of a hypothetical test where each item is scored either one 














or zero. 
TABLE I 
Items 
Student 1 s+ @ n Scores 
1 t, 
2 t, 
3 t; 
k ty 
n k 
Totals P, DP, P, Pr 2P,= 2; 


t=1 t=1 





The sum of squares “among students” is 


k 
Le ee cf a 
 .—_— =." 
The sum of squares “among items” is 
1 n ( >> ti)? 
tee? (2) 


The total sum of squares is 


k k 
(>t) (nk —- > ti) 
i=1 i=1 
nk r 
or simply the number of correct responses times the number of 
incorrect responses divided by the total number of responses, nk. This 


last result can be verified as follows. 
Let n, equal the number of correct responses and 7, equal the 


m,(1) + ”,(0) Ree 
N, + Ne 


(3) 








number of incorrect responses. Then the mean is 


pos, aa 
+2, 


oe ye ny, ~9\= mm tte oa 
NM, + Np N, + Ny 


The sum of the squares of the deviations from the mean is 





(n, + 2)? 2, + 1% 
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In terms of the notation of the analysis of variance problem above, 
this sum is clearly (3). 

By subtracting the “among students” and the “among items” 
sums of squares from the total sum of squares, we have left the 
residual sum of squares which is used as the basis of estimating the 
discrepancy between the obtained variance and the true variance. This 
estimate of the discrepancy is a better one than that obtained by 
dividing the test into odd and even halves because in the latter case 
the particular split of the test, which is only one of many possible 
ways of splitting a test, may be an unlucky division and may result in 
either an overestimate or an underestimate of the coefficient of relia- 
bility. Furthermore, it has been shown, as the reader will see in 
Part II, that the particular estimate of the discrepancy between the 
obtained and the “true” scores is the best linear estimate where 
“best” is considered in the light of the least squares’ criterion. Hence, 
it is clear that this method of estimating the reliability of a test gives 
a better estimate than any method based upon an arbitrary division 
of the test into halves or into any other fractional parts. From a 
practical point of view, the labor involved in computation is not ex- 
cessive although probably not much less than is required for the com- 
putation by split-halves. The data needed to compute the reliability 
are the number of correct responses to each item and the score for 
each subject. The use of the item counter on the International Scor- 
ing Machine makes it possible to obtain these item counts immediately. 

The method of obtaining the coefficient of reliability will be 
illustrated by a specific example. A test of 250 items was administered 
to 33 students in a class in botany in the College of Pharmacy of the 
University of Minnesota. 


TABLE II 


Analysis of Variance of Test of 250 Items in Botany, 
Administered to 33 Students 





Source of Variation d.f. Sum of Squares Variance 








Total- ------------- 8249 2002.42533 24275 
Among Items - - - - - - - - - - 249 593.81927 2.38482 
Among Individuals - - - - - - - 32 82.82933 2.58842 


Remainder - - - -------- 7968 1325.77673 16639 





The coefficient of reliability of the test may be expressed 


__ 2.58842 — 16639 _ 


tt 3 5842 986 . 
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The standard error of measurement is \/13825.77673/32 or 6.44. 

That measure of reliability which Jackson* has called the sensi- 
tivity, y, can be obtained immediately as, 

> 242203 _ as 
Sirti 14.556 and y=3.82. 

y is interpreted on the normal probability scale as follows. If y= 
2.57 for example, using the obtained scores as estimates of a certain 
capacity of the individual, we would expect to make an error as great 
as or greater than one standard deviation (of the true scores) only 
once in a hundred times. 

It may be interesting to some who are familiar with the work of 
Kuder and Richardson; that the foregoing method of estimating the 
coefficient of reliability gives precisely the same result as formula (20) 
of their paper. This fact can be easily verified algebraically. 

If two or more equivalent forms of the test are administered to 
the same group of students it is easy to extend this procedure so that 
it is possible to separate out another source of variation, the “between 
forms” variance which might be considered to be due to practice effect. 

More extended examination of the “among items” variance would 
make it possible to decide on heterogeneity of the respective difficul- 
ties of the items while a more extended examination of the “among 
students” variance would make it possible to answer certain pertinent 
questions regarding the individual differences among students. 


II 


The method used in developing the formulas for estimating the 
reliability of a test by means of analysis of variance as described in 
Part I is essentially the method suggested by Johnson and Neymant 
and later used by Jackson§. This particular approach does not give 
new or different results for the problems of tests of significance but 
does possess considerable advantage in attacking problems of estima- 
tion. The problem of chief concern here is clearly a problem of esti- 
mation, since only rarely would the reliability of a test be so low that 
its difference from zero would be a major issue, although this could 
be tested by the use of the variance ratio [(m — 1)S, — S,]/S,. (See 


* Jackson, Robert W. Reliability of mental tests. British Journal of Psychol- 


ogy, 1939, 29, 267-87. 
+Kuder, G. F. and Richardson, M. W. The theory of the estimation of test 


reliability. Psychometrika, 1937, 2, 151-60. 

t¢ Johnson, Palmer O. and Neyman J. Tests of certain linear hypotheses and 
their application to some educational problems. Statistical Research Memoirs, 
1936, 1, 57-93. 

§ Jackson, op. cit. 
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equations (9) and (17) for definitions of these quantities). 

Assume that the score, X;,, of the s-th student on the i-th item of 
the test may be represented as the sum of four independent factors 
or components (i.e., the vector X;, is resolved into four mutually per- 
pendicular components). These four components may be described as 


follows: 


(1) a component common to all individuals and to all items; 

(2) a component associated with the item ; 

(3) a component associated with the individual ; 

(4) an error component that is independent of (1), (2), and 
(3) but includes a multitude of small variations produced 

( by a multitude of causes which, though each by itself is 
relatively unimportant and unpredictable, taken together 

* may be thought of as being distributed normally with vari- 
ance o”, the precise value of which is unknown. 


This may be expressed in mathematical form by assuming that 
Xis—AtK + De + Vis, 


where 7 = 1, 2,---, nm and s=1, 2, ---,&, represents the score of the 
s-th individual on the i-th item of the test. Here A is the component 
common to all individuals irrespective of the particular item under 
consideration; it will be shown to be a constant for all students and 
all items. In the foregoing expression for X;,, ¢t; is the component 
associated with the item and p, is the component associated with the 
student. (It will be seen later that (A + #;) is the quantity often called 
the difficulty of the item.) If a student were an “average student” 
his “true score” on the i-th item would be (A + #;) ; however, if he 
were above “average” his “true score” would be higher than (A + ¢;), 
such as (A +4;+,). (If he were below “average” p, would be 
negative. ) 

We must further assume that the error component, y;,, of the 
i-th item is normally distributed with the same variance, o”, as is the 
error component of every other item, and that y;, is independent of 
Yje Where i # j. 

Then, since 


Yis = (Xis ~ i ti — Ds) (1) 


is distributed normally with standard deviation, o , 


Me 


n ek 
2Y¥is= = my tae A § )* 


t=1 8= 4=1 8= 


is distributed as ’. 


" 
- 


~ 
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Hence 
» & 
Cee hee Ah Bed” (2) 
i=l = 

The reliability coefficient of a test is the ratio of the variance of 
the “true scores” to the variance of the obtained scores, or in other 
words, gives the percentage of the obtained variance that may be 
spoken of as “true” variance or not due to the unreliability of the 
test. If we let o*; represent the variance of the obtained scores and 
o@ the discrepancy between the variance of the obtained scores and 
the variance of the “true” scores, the reliability coefficient is given by 
the ratio: 7; = (07; — oa) /o?;. The variance of the error term, yi, , 
or o7;, is the expression of which we wish to obtain the best estimate. 
According to Markoff’s theorem,* the best linear estimate of yi, can 
be obtained, in the situation where the o’s of the independent observa- 
~tions are all equal, by minimizing the sum of squares (2) with respect 
to A, t;, and p, as independent variables and substituting these values, 
A*, t;°, and p,°, in (1) to give yi,', the best linear estimate of the 
discrepancy between the obtained score and the “true” score. 

A first necessary condition for minimizing (2) is that the partial 
derivatives with respect to A, ¢; and p, must vanish. 


0x? n &k 





yi ie (44 —-A-t:—-Hi); (3) 
> POS an 
06, exe = ¢ is i Ds) 

for each i=1, 2,---,”; (4) 
07 





=-23 (Xu-A-t—p,) 


for each s=1, 2,---,k. (5) 


Setting each of these partial derivatives equal to zero and solving 
simultaneously gives the values for A, t;, and p, which minimize 7’. 
These values of A, t;, and p, which render y* a minimum will be 
designated by A°, ¢;°, and p,°. 


1 an &k 1 n 1 k 
A=—-3 DXu-— Ste — Sp; (6) 


nk i=1 8=1 nj 1 
ee. ; 
{eae Xe ~ 9) — a G1,8,--59); 


* Neyman, J. The Markoff Method and Markoff Theorem on Least Squares. 
Journal of the Royal Statistical Society, 1984, 97, 598-594. 
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=X (Xi,— 4°) —A®° (s=1,2,-:-,k) (8) 


i=1 


Substituting these values of A°, ¢;°, and p,° in (2) gives the 
minimum value of y? which is designated by S,. 


n &k 
S=z - (Xie ok. i t,° rr p.°)* 


n k —_ poe tas 
=z = (20 — 2. — 2, + 2)* ’ (9) 
where 
_ Ast 7 oR 
a ee Se at eae is 5 a Xis. 1 
" 72x ~ Ex i nk 2 = (10) 


Substituting these values of A®, ¢;°, and p,° in (1) gives the best 
linear estimate of y;, (i.e., the error component of the response of the 
s-th student to the i-th item). 

Thus 


(11) 


8! 


Y' is = Xi, — Yi. — Xs i 


’ 


n ik n &k 
Since Y Syn =0, S=Z SZ (y's — y)*, 8 that S, is f times 
4=1 8=1 é=1 8=1 
the variance of the y’;,, the error component, where f is the number 
of degrees of freedom or the number of independent variates necessary 
to express the sum, S,. It is clear that f= (n—1)(k—1) if we 
consider the nk X;,’s arranged in n rows and & columns, so there are 
(n — 1) independent variates in each of the (k — 1) columns. Hence 
the best estimate of the variance of the error component is 


So 
(n —1)(k— 1) © 
In order to determine the best estimate of o?, , the variance of the 
component p, associated with the student, assume p, to be normally 
distributed with variance o”,. Since p, is independent of yi; , 
n k 
= D (vis + ps)? is distributed as 7. 


$=} a1 


Then 


(12) 





=> (X;,, — A — &)*; (13) 


i 


iM-e 


& 


Ot 23 3 (X,-A-t); (14) 


#=1 8=1 
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2 k 
os =-23(Xu—-A-—t) (4=1,2,---,9.) (15) 
i s=1 ; 
Setting each of these partial derivatives equal to zero and solving 
simultaneously gives the values of A’ and t’; which, when substituted 


n k 
in (13), will give © © (yis + p.)? its minimum value which is desig- 
i=1 8=1 


nated here as S,. 
Then 
n ik i 
s,=2 py (Xsq ~~ &.)*. (16) 
t=1 8=1 
This sum of squares gives the basis for estimating the variance 
o,”, since (S, — S,) is the “among students” sum of squares. 


Let 
S,=S, — So. (17) 


Then i ci i is the obtained variance “among students.” The ob- 
tained variance “among students” is not the best estimate of the true 
variance in case the hypothesis of homogeneity of the sample has been 
refuted as will usually be the case in test item responses. Instead, ac- 
cording to the work of Irwin,* the best estimate of the variance 
“among students” is obtained by subtracting the estimate of the er- 








: So ” : ze 
ror variance NE a from the obtained variance “among 
students.” 
Thus 





S, So 
a1 (k—1)(n—1) (1 
is the best estimate of the true variance of the students’ responses. 
Hence the percentage of the obtained variance that is not associated 
with unreliability of the test is 














ict: %, 
niga: (k-1) (k—1)(n—1) (19) 
S: 
k-1 
or 
(n os 1) S; red So 
=). iad (20) 


Formula (19) is the one used for the computation of estimated 
test reliability in Part I. : 


* Irwin, J. O. Mathematical theorems involved in analysis of variance. Jour- 
nal of the Royal Statistical Society, 1981, 94, 284-800. 
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A FACTORIAL STUDY OF NUMBER ABILITY 


CLYDE H. COOMBS* 
UNIVERSITY OF CHICAGO 


In order to investigate certain hypotheses concerning the nature 
of number ability, and, secondarily, the nature of perceptual speed, 
a battery of thirty-four tests was given to 223 Chicago high school 
seniors and the data were factored by the centroid method. Seven 
primary factors were identifiable upon rotation. Several deductions 
are made relative to the interpretation of the factors and relative to 
poe Haare an of the data with the hypotheses which were to be 
tested. 


I. Introduction 


Since the development of multiple factor methods by L. L. Thur- 
stone, at least nine multiple-factor studies of cognitive abilities have 
been carried out in his laboratory. The general results of these stud- 
ies have been in very high agreement. The same factors have occurred 
repeatedly in different test batteries with subjects from eighth-grade 
level to college freshmen. One of the clearest of the primaries and one 
that has been repeatedly obtained is number ability, characteristically 
defined by tests of addition and multiplication. These two tests are 
referred to as “simple” tests in that they have a large projection on 
only one or only a very few common factors. 

It is not very satisfying to define an ability in terms of the con- 
tent of the tests which measure that ability. This procedure should 
be regarded as a makeshift until investigations may be carried out 
which give insight into the psychological nature of the fundamental 
process characterizing the ability. For example we may ask: Is the 
ability defined by addition and multiplication specific to numerical ma- 
nipulations or does it represent some more fundamental unity that 
transcends numerical operations? Is it inherent in the organism or 
a product of training? With the idea of gaining some insight into 
these questions we shall review some of the facts gained from case 

*I wish to express my great appreciation of the aid of Professor L. L. Thur- 
stone whose generosity made this study possible. Grateful acknowledgment is 
made of the aid of Mr. Ledyard Tucker in the use of the I.B.M. machines for ob- 


taining the intercorrelations and the centroid factor loadings, and to both him and 
Mr. Harold Bechtoldt for aid in the testing of subjects. 
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histories of arithmetical prodigies and from the functional analysis 
of cases of acalculia or arithmetic disability. 

Prior to the isolation of number ability by multiple factorial 
methods, there have been indications that this ability represents a 
functional unity in the mind. Scripture* in his article on arithmetical 
prodigies gave a brief account of more than fifteen individuals, dat- 
ing from about 100 A.D., who gave evidence of phenomenal reckoning 
powers. Mitchell+ also summarized what was known about mathe- 
matical prodigies and gives a brief account of his own case. Other 
psychologists have contributed to the knowledge of reckoning ability 
by psychological studies of individual cases.t Intensive experimental 
investigations of individual cases are lacking, most of the literature 
being biographical in nature. However, certain things which are of 
interest here seem to be clear. Mitchell states, “Skill in mental calcu- 
lations is... . independent of general education; the mathematical 
prodigy may be illiterate or even densely stupid, or he may be an all- 
around prodigy and veritable genius.”§ With the possible exception of 
memory for numbers, case histories of rapid mental calculators indi- 
cate that these individuals do not necessarily show any unusual abil- 
ity in any other line of endeavor. Their most common characteristic 
is an unusual interest in and liking for numbers for their own sake. 
This interest is accompanied by a great deal of practice in their ma- 
nipulation. 

These data on mathematical prodigies seem to indicate that edu- 
cational forces are of slight importance to this ability. A large pro- 
portion of the cases cited had little or no formal education and gave 
evidence of their unusual ability at a very early age. It might be 
argued that their interest and liking for numbers and numerical ma- 
nipulation caused them to practice a great deal, and that this is essen- 
tially the counterpart of modern education. This may be true, but it 
seems more reasonable that an inherently superior native ability gave 
impetus to the interest, rather than vice versa. It is very unlikely 
that an average, or even below average, child could, by reason of in- 
terest, develop an ability to do numerical calculations which would 
transcend the ability of all his contemporaries. 


. we E. W. Scripture. Arithmetical prodigies. American J. Psychol., 1891, 4, 
ion {Frank D. Mitchell. Mathematical prodigies. Amer. J. Psychol., 1907, 18, 
-1438. 
tA. Binet. Psychologie des Grands Calculateurs et Joueurs d’échecs, Paris: 
Hachette, 1894, pp. 364 ff. 
sea — and A. L. Bryan. An arithmetical prodigy. Psychol. Review, 
. , 1 . 
$F. D. Mitchell, op, cit., p. 181. 
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Studies of individuals with marked arithmetical disability are 
also very suggestive. Two cases have been reported in detail by Wer- 
ner and Straus.* Case I revealed a specific deficiency in the finger 
schema and a general inability to grasp relations in space. Case II 
revealed a specific disturbance in the comprehension of visual pat- 
terns and in general a deficiency in grasping simultaneously the mul- 
tiple aspects of a visual unity. All of these disturbances were found 
to differentiate a group of subjects selected for poor arithmetic ability 
from a group of subjects selected for good arithmetic ability. Gutt- 
mann, summarizing the evidence for congenital arithmetic disability 
arising from structural or functional anomalies of the brain, states, 
“There is little doubt that focal brain lesions may produce acalcu- 
Pe 
The evidence from such cases of deficiency in number ability 
tends to indicate that the deficiency is more fundamental than mere 
lack of ability to manipulate numbers. This would be the case if the 
psychological process which we call number ability were inherent in 
the organism. Numbers, as a cultural invention, would be a much 
later development than a mental ability evolved by the slow processes 
of evolution. 

It is reasonable, therefore, that tests like addition and multipli- 
cation really measure some more fundamental process than mere num- 
ber manipulations per se. Hence it should be possible to devise tests 
of a non-numerical character which would measure this fundamental 
process. To do this one must make an hypothesis as to the nature of 
this process characterizing number ability. 

If it is correct to assume that two senior students in high school 
know equally well that, say, eight and six are fourteen, and similarly 
for other possible number combinations, then the difference between 
their scores on a test of addition represents a difference in the rapid- 
ity with which the individuals can recall and manipulate these well- 
established associations. One might call this an agility in manipulat- 
ing a symbolic system according to a specified set of rules, with two 
restrictions, that the symbolic system be familiar and that the rules 
be highly practiced. In a loose sense it might be descriptively referred 
to as mental agility, though this term would have more extensive im- 
plications than are desired here. 


* Henry Werner and Alfred Straus. Problems and methods of functional 
analysis in mentally deficient children. J. Abn. & Soc. Psychol, 1939, 34, 37-62. 
E. Guttmann, Congenital arithmetic disability and acalculia, British J. med. 
Psychol, 1937, 16, 18. 
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If number ability were basically such a process as this—the 
quick recollection and manipulation of well-established associations— 
then this fundamental process would transcend numerical operations. 
Inasmuch as “well-established associations” are products of educa- 
tional and environmental forces, such forces would obviously play a 
role in the establishment of the ability. In view of the fact, however, 
that the learning curve has a horizontal asymptote, it is possible to set 
up tests of a non-numerical character in which the associations to be 
established are so simple that the asymptote is approached very early. 
These new tests would then measure primarily the rapidity with which 
these “well-established associations” could be manipulated. 

From this hypothesis as to the nature of number ability certain 
deductions may be made which are susceptible to experimental veri- 
fication. Briefly these may be stated as follows: 

1) Tests involving new rules for manipulation of a symbolic 
system are better tests of number ability after practice than they are 
before. 

2) Of several tests involving manipulation of a symbolic system, 
the one in which the symbolism is more familiar provides the better 
measure of number ability. 

Another hypothesis as to the basic process involved in number 
ability may also be tested. It has been suggested by H. D. Landahl 
that the operation characterizing the tests defining the number factor 
is in the nature of a serial response, in that each response to a pair of 
numbers leads to the next response. This may be submitted to experi- 
mental verification by varying the length of the addition or multipli- 
cation problems. If the serial response hypothesis of number ability 
is correct, the addition of two digits would not be a measure of num- 
ber ability, but the addition of four digits would. 

Inasmuch as the tests of Identical Forms and Verbal Enumera- 
tion have not been very satisfactory as measures of perceptual speed, 
several cancellation tests were incorporated in the battery to see if 
they might not give a better definition of this factor. This clarification 
was of particular interest because of the fact that in a very simple 
number problem such as the addition of two digits, perception of the 
numbers and knowledge of the answers seem almost simultaneous. 
With good tests of the perceptual speed factor it is possible to deter- 
mine what relation, if any, exists between it and number ability. 

Tests were designed to check the validity of the hypotheses as to 
the nature of number ability and to determine what relation number 
ability bears to perceptual speed. These tests were incorporated with 
a variety of other tests covering the domain of known mental abilities, 
and the entire battery was analyzed by Thurstone’s centroid method. 
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II. The Experiment 
THE SUBJECTS 


In the spring semester of 1939, the Chicago high schools offered 
the members of their senior classes a course in self-appraisal. Of their 
own volition the students in these classes took personality and apti- 
tude tests with the intention of gaining some evaluation of their ca- 
pabilities. Because the subjects for this experiment were taken from 
six such classes in six south-side high schools, it may be assumed that 
their motivation was fairly good and fairly constant. The six schools 
and the number of complete records obtained from each are as follows. 


Englewood High School - - - - - - - - - 87 
Fenger High School - - - - - - - - - - 84 
Hirsch High School - - - - - - - - = = 28 
Hyde Park High School - - - - - - - = - 21 
Morgan Park High School - - - - - - = - = 


Parker High School - 
Total - - - - - - - = = = 223 
Each class was visited for approximately an hour on each of five con- 
secutive days, and from three to five tests were given to the subjects 
at each session. 
TABLE 1 
Tests for the Primaries 











Number Test Primary 
19 Identical Forms - - - - - - - - P 
20 Verbal Enumeration- - - - - - - P 
21 Addition - - - - - - - - - - N 
22 Multiplication - - - - - - - - - N 
23 Completion - - - - - - - - = = V 
24 Same-Opposite - - - - - - - - Vv 
25 Camdes sare OS ere ee Ss 
26 Figures - - -----+-+--+--s s 
27 Initiale - - - - ---+-+-+-s M 
28 Word-Number - - - - - - - - - M 
29 Letter Grouping - - - - - - - - I 
30 Marke - = --+-+-+-+-+++e-s I 
31 Number Patterns - - - - - - - I 
32 Arithmetic - - - - - - - - - - D 
33 Number Series - - - - --- - - D 
34 Mechanical Movements - - - - - - D 
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THE TESTS 


The tests used in this study may be divided into two groups: 1) 
the tests for the primaries and 2) the experimental tests. 

The tests for the primaries, listed in Table 1, consisted of the 
sixteen tests in the American Council on Education Tests for Primary 
Mental Abilities, experimental edition, 1938. These tests were given 
by Miss Ruth Wright, of the Bureau of Child Study, Board of Educa- 
tion of Chicago, as part of the regular testing program of the high 
schools. 


The experimental section of the battery consisted of eighteen 
tests selected and designed to answer specific problems. Each test is 
described below together with a statement of the purpose for which 
the test was designed. 

Two-Digit Addition (1).—This test and the following two were 
designed to investigate the hypothesis that the number process was es- 
sentially in the nature of a serial response. If the hypothesis is cor- 
rect, the saturations of tests (1), (2), and (3) on the number factor 
should progressively increase. ‘The fore-exercise of this test, the 
whole of which consisted of 238 problems, was as follows: 

Look at the following addition of problems. 


Se 22.8 33 8 e SF ee as 
So ca @. oe ye. wo 


Because they are wrong two of the answers are un- 
derlined with an X. 

In the following problems cross out every answer 
that is wrong. 


2s 8 wt Mw? 1 oe WY 


Three-Digit Addition (2).—This test consisted of 170 problems. 
The fore-exercise follows. 
Look at the following addition problems. 


S-s+s3 4 -% @ @ 8 2 es: Sc 

7. © 3 Bs ae Bees 

G8, ek: oe a OE ne ee ae 

21.0609 61T 8 SS PY 8" 6 22 
xX > ee x 
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Because they are wrong four of the answers are un- 
derlined with an X. 
In the following problems cross out every answer 
that is wrong. 
78 @°8- 9-5 
Gt Le ieee soe: WS. OZ Sr oe. Be 
it es ee 8 


16 18 17 14 18 15 14 12 15 14 19 14 


Four-Digit Addition (3).—This test consisted of 153 problems. 
The fore-exercise follows. 
Look at the following addition problems. 


28-38" 022 16 EE 2S 2k eS “SE 27 
x x x x 


Because they are wrong four of the answers are un- 
derlined with an X. 

In the following problems cross out every answer 
that is wrong. 


19 17 19 20 25 19 19 24 14 22 12 18 


AB (4).—The AB and the ABC tests were designed to contrast 
with the Forms test to check the hypothesis that of several tests in- 
volving manipulation of a symbolic system, the one in which the sym- 
bolism is most familiar provides the best measure of number ability. 
The familiar alphabetical character of the elements in the two former 
tests contrasts markedly with the unfamiliar character of the elements 
used in the Forms test. Also the greater simplicity of the AB test com- 
pared to the ABC test would throw some light on the extent to which 
the speed element characterizes number ability. The fore-exercise of 
the AB test follows: 


Only three letters, A, B, and C, are used in this test. 
They are combined in various ways, but there are 
only two rules. The rules are: 


(1) A combination of any two different letters is 
equal to the third letter. 
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Examples: AB=C, AC=B, BC=A, 
BA=C, CA=B, CB=A. 


(2) If a letter is combined with itself, the com- 
bination is equal to that letter. 


Examples: AA=A, BB=B, CC=C. 


Below are some problems for you to practice on. If 
the answer to a problem is A, put a check (V) in the 
first column, if B put a check (Vv) in the second 
column, if C put a check (Vv) in the third column. 
The first three problems have been worked for you. 
Work them yourself to see that they are right 
and then go on with the others. 


ABC ABC ABC 
Weta Beas. epee 
aac... aiecsncx. salen 
A oe ee oe 
7 a ra + ervaenne 


This test consisted of 210 problems. 
ABC (5).—This test consisted of 60 problems. The fore-exercise 
follows. 
Only three letters, A, B, and C, are used in this test. 
They are combined in various ways, but there are 
only two rules. The rules are: 
(1) A combination of any two different letters is 
equal to the third letter. 


Examples: AB=C, AC=B, B 
BA=C, CA=B, C 


(2) If a letter is followed by itself, the combina- 
tion is equal to that letter. 


Examples: AA=A, BB=B, CC=C. 


The example below has been worked out according 
to these rules. 


C=A, 
B=A 


ABA=B 


Here is the way you solve the problem. Combine the 
first two letters. The combination A B = C. Then 
combine this C with the next letter, which is A. The 
combination C A = B. 

Work the following examples and write the answers 
in the blanks. 


CBB=— CAC= 
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You should have written C in the first blank and A 
in the second blank. 
Here is a longer example which has been worked 


out. 
CABC=A. 


The first two letters, C A, equal B. This letter com- 
bined with the next gives B B, which is equal to B. 
This letter, combined with the next, gives B C= A. 
Find the answer to the following example and write 
it in the blank. 


BCAC=— 
You should have written B. 


Here are some more problems for you to practice 
on. Write the answers in the blanks. 


BCCACS=— JARS A=— ACKICA=— 


Forms (6).—This test was of exactly the same nature as the ABC 
test except that three non-meaningful geometrical designs replaced 
the letters A, B, and C. 

Alphabet I (7).—This and the following two tests were designed 
to check the hypothesis that practice with a set of rules would im- 
prove a test as a measure of number ability. If the hypothesis is cor- 
rect, the three tests, Alphabet I, Alphabet II, and Alphabet III should 
show progressively increasing validity as measures of number ability. 
The fore-exercise of Alphabet I follows. 


In this test the letters of the alphabet are combined 
in various ways according to three rules. 

(1) In a combination of two letters like MP, the 
letters are in the same order as they occur in the 
alphabet and there are several letters between them, 
N and O. Two letters like RT are also in alphabeti- 
cal order and have one letter between them, S. Simi- 
larly one could write other combinations of two let- 
ters in alphabetical order in which there are three 
or more letters between them. 

A combination of two letters in alphabetical order 
which has letters between them is equal to the letter 
in the alphabet which follows the second letter in 
the combination. For example, the.two letters in the 
combination MP have letters between them, so the 
combination is equal to Q because Q follows P in the 
alphabet. Similarly, RT = U because the letters RT 
have a letter between them and U follows T in the 
alphabet. 

Other examples: CE= F, HK=L, DV=W. 

(2) In a combination of two letters like PM the 











PSYCHOMETRIKA 


letters are in backward order and have several let- 
ters between them, for example, N and O. Two let- 
ters like TR are also in backward order and have 
one letter between them, S. Similarly, one could 
write other combinations of two letters in backward 
order with three or more letters between them. A 
combination of two letters in backward order which 
has letters between them is equal to the letter in the 
alphabet which precedes the second letter of the 
combination. For example, the two letters PM have 
letters between them, and so the combination is 
equal to L because L precedes M in the alphabet. 
Similarly, TR = Q because the letters TR have a 
letter between them and Q precedes F in the alpha- 
bet. 


Other examples: 
HC=B, VN=M, JH=G. 
(3) A combination of two letters which have no 


letters between them is equal to the second letter of 
the pair. 


Examples: OP=P, GH=dH, RS=S, 
PO=0, HG=G, SR=R. 
Here are some problems for you to practice on. The 
first few have been answered for you. Check them 


to see that they are right and then go on with the 
others. Write the answer in the blank space. 


DF=G JL=M RS= FH>= EC= 
GJ=K TS=S LM= BI= YF= 
RI=H WY= UT= HJ= OL= 


This test consisted of 192 problems. , 
Alphabet II (8).—Consisting of a second page of problems, this 


test was similar to Alphabet I. The subjects upon finishing Alphabet 
I had a rest period of several minutes and then took Alphabet II. The 
test consisted of 192 problems. 

Alphabet III (9).—This test consisted of 192 problems similar to 
those in Alphabet I and Alphabet II. There was a brief period of rest 
between the tests Alphabet II and Alphabet III. 

Digit Cancellation (10).—In conjunction with the next two tests, 
digit cancellation was designed to clarify the definition of a percep- 
tual speed factor and thereby indicate what relation this factor would 
have to the number factor. The fore-exercise of this test was as fol- 
lows: 
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Look at the rows of numbers below. A parenthesis 
has been put around each number 5. 

Sheer Hey Fs —v 

J te oe Se ee 

a pi eS GS ae 

4 6 9 (5) (5) 0 


Put a parenthesis around each number 5 in the fol- 
lowing rows. 


oo 00 & 
Aone 
oaone 
one oO 
anon 
bo bo by © 


You are given several longer rows of numbers be- 
low. Put a parenthesis around each number 5. 


ss O'S 6 2it 6 s°<4 Tt-9 8 
Scattered X’s (rowed) (11).—This test consists of ten pages of 
letters in rows through which x’s were scattered. The fore-exercise 
is below. 
Look at the rows of letters below. A parenthesis has 
been put around each letter “x.” 


Z b s h s g 
in +&..4 eee -¥ 
1 m v_ (x) (x) o 
e (x) d f k a 


Put a parenthesis around each letter “‘x” in the fol- 
lowing rows. 


mbtjegr 
aexdvx 
©-x To ft h 
ok eS SE 


You are given several longer rows of letters be- 
low. Put a parenthesis around each letter “x.” 


vutoecmeseefxemis:sq its 


Scattered X’s (pied) (12).—This test consists of seven pages of 
pied letters with seven x’s on each page. The subjects were instructed 
to find as many x’s on a page as they were able and then to turn to 
the next page. 

Identical Numbers (13).—This test, in conjunction with Highest 
Numbers and Digit Cancellation, was selected for the purpose of de- 
termining whether tests involving numbers and numerical concepts 
necessarily had a projection on number if they did not involve ma- 
nipulation of the numbers. The test, the fore-exercise of which fol- 
lows, consists of 45 columns with 29 three-digit numbers in each. 
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The number at the top of the first column of figures 
is 634. A mark has been made under each 634 in 
the column. In the second column, a mark has been 
made under the 876, because 876 is the number at 
the top of that column. In the third column, the two 
795’s have been marked, because 795 is the number 
at the top of the third column. 

The number at the top of each of the other columns 
is repeated one or more times in that column. Find 
those numbers as quickly as possible and put a 
mark under each of them. Go right ahead. 








634 876 795 423 279 874 
693 648 583 837 363 282 
850 328 795 115 643 663 
634 932 189 423 279 539 
518 879 342 528 375 314 
398 875 795 969 470 375 
696 470 896 274 887 576 
634 697 247 423 699 874 
574 876 319 627 291 850 
628 294 468 423 983 677 
634 982 543 962 585 846 








Highest Number (14).—This test consists of 80 columns with 40 
three-digit numbers in each. The fore-exercise is as follows: 


You are given five columns of numbers below. Find 
the highest number in each column and put a paren- 
thesis around it. The first two columns have been 
done correctly. You mark the other three. Go right 
ahead. Do not wait for any signal. 











1 2 3 4 5 
298 189 229 189 149 
142 237 340 376 580 
389 (746) 187 234 689 
527 642 246 427 827 
462 248 546 167 486 
127 548 827 349 682 
482 329 628 256 595 
(536) 735 821 453 337 
227 670 342 118 428 


162 167 119 297 821 





Size Comparison (15).—It was thought desirable to set up a test 
of a “quantitative” nature but not involving numerical concepts to 
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see if number ability had a characteristic of quantitative thinking. 
The test consists of 69 items. The following is the fore-exercise. 
Look at the following pairs of words. 


sardine —(s) hark 
(G)ermany —Cuba 
(P)lanet —palace 


A parenthesis has been put around the first letter of 
the word in each pair which means the larger of 
the two things. A shark is larger than a sardine; 
Germany is larger than Cuba; a planet is larger 
than a palace. 

In the following pairs of words put a parenthesis 
around the first letter of that word in each pair 
which means the larger of the two things. 


Texas —Maine clock —watch grape —lemon 
thumb —foot tree —forest wrench —nut 
inch —mile rock —pebble cup —barrel 


Substitution I (16).—This test, in conjunction with Substitution 
II and Substitution III, was designed to see if an increasing familiar- 
ity with the translation of an arbitrary symbolism would have any 


TABLE 2 


Time Limit and Scoring Formulae used on the Experimental Tests 








Number Test Time Limit in Minutes Scoring Formula 





Fore-exercise* Test 





1 Two Digit Addition - - 1 4 R-W 
2 Three Digit Addition - 2 4 R-W 
3 Four Digit Addition - - 2 4 R-W 
4 AB ----+-e- = 3 3 2R-W 
5 ABC - - - - - = = 8 5 2R-W 
6 Forms - - - - - - 13 5 2R-W 
7 Alphabet I- - - - - 10 10 R 
8 Alphabet II- - - - - 5 R 
9 Alphabet III - - - - 5 R 
10 Digit Cancellation - - 2 4 R 
11 Scattered X’s (rowed) - 1 4 R 
12 Scattered X’s (pied) - 2 4 R 
13 Identical Numbers - - 2 8 R 
14 Highest Number- - - 2 3 R 
15 Size Comparison - - - 2 LT R-W 
16 Substitution I- - - - 10 10 R 
17 Substitution II - - - 5 R 
18 Substitution III - - - 5 R 


* The time limit for the fore-exercise was not rigidly adhered to except in the ‘case of the sub- 
Stitution tests. 
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significance in relation to number ability. This test and the two fol- 
lowing tests each consist of 90 words in code which are to be trans- 
lated. The same code is used throughout and is given at the top of 
each test sheet. 

Substitution II (17).—This test consists of a second page of code 
words similar to Substitution I and of the same length. The code is 
repeated on this test blank. 

Substitution III (18).—This test is a third page of code words 
which are similar to those of the two preceding tests. The same code 
is also repeated on this test blank. 

The time limit and scoring formula used for each of the eighteen 
tests comprising the experimental section of the battery are given in 
the following table. 


£00 - 
critical value 


a ee ddd A 


isi 
“Fr 
350 
soo} 
a 
wil. 
150F 


100 F 








l 1 l l l l l l 1 j 
0 1 2 3 4 5 6 é 8 9 10 


nth factor residual table 
Fic. 1—Number of negative signs in n‘” factor residual table after sign change. 


THE FACTOR ANALYSIS 


In order to obtain the product moment correlation coefficients 
between tests by means of the punched card technique and I. B. M. 
machines, it was necessary that all test scores be expressed in no 
more than two digits. For this reason the scores of certain of the 





its) 
- 
re 
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ste 918 L982 


OLO 





cod 890 288 LUE L2e G6Z IST 99E 3222 OZ LEI O22 BLS 92E GEL SHS GIS SBI SBI GIF HOS SHE G9T B92 19% OTS 602 FOT HE 
969 292 868 882 960 89% Sr Zor PSs Sse 18 FLe 80S LOT 890 OO FLT 998 22 SHIT IST 680 991 LOF SLY O8F BIZ 628 OZE BSE T9S 892 $B 
10 OF8 SBT O9T L192 68S ZEEE PSE SSS PET LES GT $60 380 960 66L 982 F9T 690 880 800 910 198 O66 OTP 88T ZIS OLZ LES 80S G9T Ze 
y88 108 180 80£ 08 S88 7290 620 ZLe PIE ZT LH HET IZT BBl IST O88 HET 002 ZHZ BB8I IhZ 922 322 09% Zl 92 BzZE 908 98% IE 
628 910 990 688 90h 998 892 8ET The 182 892 9PZ 09% IE OTS Ih2 LET 860 SIT 691 SLh O8h 80S OIh P68 ses EZ 09% SLT O08 
190 26I 882 92 962 8IZ 08% 962 G12 60S O99T TLI 192 BLT 892 692 802 ITS LZB E2h SSh Zr LIS BOE T68 26Z LES 922 62 

098 920 8h0 $890 800 O8f 991 280 80 982 LOZ FEZ Z2T 980 SZI 800 280 080 SrT TOT OFT S8T LOT LOZ SEL SIT SIT g% 
SvT OFZ O6T SOT 802 OST 92T GIT LHT L60 002 LOT PZT LOT S80 LLO OTT 982 FHLZ 99% TILT ZS OLT SLT 82s Sat Lz 
v8L 62 IIS SLT P8l 882 91S LPT 9ST 8S IS8 HES 8BS0 SLI 892 QLT PSS SLE LIb 90S SHE BIE 622 99% ZIZ 9% 
22z 982 LET OTS 022 OL SOT G6HT 682 992 692 HEI SZI 922 BIZ LOW SIb Shh LOS 188 SHE ZZ SS% SLI GZ 
60L 902 Z6I 89h T9T P80 060 OFT 8LS $92 ZHZ ZIO- 100 I8I I19 L2G P8b LLZ Bh OFF SZE HES 692 FZ 
6VI I2E 998 Z9T 900 280 860 289 LET 9ZI 260 O10 SI Sah HHH SEH OFZ BES 128 SZ 99% LST 
98S 828 122 19% LOT 98S SIZ 998 GOP THT 26T PSE LES Seg SHE F822 BSE BOF 6E9 SOL SEL Zz 
Sv§ LB 8s SIZ 9S GLE BEF BSF OPT LLZ 06E TOS 88% 992 GSS ILE 968 9F9 LEO GIS 12 
008 GLI SOT Z6T 989 182 SSh OLT 86% 898 FOS B08 BIS 988 VIF HOF OTE LIE HEE 02 
992 982 FOE 162 HE G62 SEZ E9F GES L182 392 88S 108 LIZ HHE SIZ 68% 9392 61 
816 888 993 HIE BPE 960 GES 28S 908 OTS IZS 26E ZEEE SOF GHZ BES LEZ 8sI 
TS8 88S 88F SPE POT 80% 802 808 FEZ SOS S2hP SIE SIE G6IZ S6T Y96T LT 
283 S88 OS$ SST PSS G92 GI BOF F2H SIb OFF SLE 9ZE 88% 8% 9 
698 ZI€ SSI Gee 6LZ ZH Soh Tbh 20S LYb PSP I6S FEE SPE ST 
99 SST LOS 82h IIb 60F F9S 898 SLE Brh ISh Ihh Z9P PFT 
vIZ OLE 809 P88 SZE BPE OLE LOE HHH EOS S6F OFS SI 
L6p 162 GLI SIZ SLT 920 180 621 6ST Z22t Set at 
602 O9T ZHI PST SIZ SLT 918 88% FLZ 892 IT 

328 628 88S 882 262 Zev Abb YFP Zab OT 

S26 906 28S 369 FH9 LOS 86h 668 6 

126 91S 168 889 68h ZLP S88 8 

ves 109 889 OLb 6hP S98 2L 

699 S19 968 S88 108 9 

199 “OLb Ssh LOP Sg 

stg cos 88h F 

988 88L § 

6r8 

T 

ye ee ae ee ee ee «| ee 2 I es eS 6 8 L 9 9 v & z if 
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tests were linearly translated to two digits, a process which does not 
affect the correlation coefficient. The correlation matrix is presented 


in Table 3. 

The correlations were factored on the I.B.M. machines by Thurs- 
tone’s centroid method. Application of a criterion developed for the 
purpose of detecting significant common factor variance in a ma- 
trix of intercorrelations indicated that ten factors were significant 
and the eleventh was not.* This criterion is based on the number 


TABLE 4 
Centroid Factorial Matrix 


i 8a a UV. OE Va x 


64 -39 24 33 18 06 O06 21 12 -12 84 
72 -27 24 33 28 10 16 11 O07 -09 90 
71 -27 20 382 25 08 17 10 11 -04 84 
72 08 -06 18 -O07 -11 -06 06 -07 12 60 
06 -04 14 -18 15 58 
62 03 -28 O07 08 -09 -12 10 -24 14 58 
77 387 -24 12 -05 -14 25 10 18 -05 92 
77 36 -24 17 -06 -14 24 11 12 -12 98 
78 34 -24 18 -07 -15 23 14 10 -08 98 
10 54 -82 138 05 -238 -21 09 -04 04 12 54 
11 42 -85 16 -26 -29 -32 04 -138 -11 08 61 
12 29 -17 14 -24 -23 -20 23 -18 -04 -10 38 











CoOoNA oR OD 
D> 
a 
ra 
© 
l 
rary 
ie\) 
_ 
~] 
o 
an 


38 52 34 20 -11 18 16 24 -15 12 11 += 60 
34 44 11 12 -26 -10 09 OF -06 11 08 838 





* A paper presenting the logic upon which the criterion is based and the criti- 
cal values for a given number of tests in the battery is in preparation. 
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of negative entries in the residual matrices after sign change. The 
critical value in the case of 34 variables is 468. When this value is 
attained no more significant common factor variance remains. The 
number of negative signs in the tenth factor residual table after sign 
change was 478. The number of negative signs in each residual table 
after sign change is presented in Figure 1. 

The loadings on the ten centroid factors are presented in Table 
4, This table was given to a disinterested person, who rearranged the 
table by rows and gave the variables a new set of numbers. The code 
remained in the possession of this person throughout the time the 


TABLE 5 
Rotated Factorial Matrix* 








2 eee oe ee se CO eS 


74 02 05 -03 -01 -09 -06 -02 03 06 
72 04 -03 02 -02 06 03 04 -02 02 
66 01 -04 03 00 O07 00 05 -O01 O07 
10 09 09 O06 O09 -06 14 24 -04 30 
04 28 27 O1 265 
02 -03 10 09 -05 -07 37 26 04 25 
-02 -03 01 03 O1 O06 -03 64 02 08 
02 00 00 04 OO -O01 -04 66 02 -01 
03 -01 00 O01 00 -02 -02 66 O01 04 





CWONAO TP CDR 
i=) 
for) 
_ 
_ 
o 
ry 
— 
| 
_ 
~ 


34-06 15 i6 -05 19 380 -09 00 15 00 





* All entries have been multiplied by 100 to eliminate the decimal. 
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rotations were being made. The centroid matrix was rotated by 
means of extended vectors and the oblique rotational method. In ro- 
tating “blindly” in this manner the only criterion determining rota- 
tions was to go strictly to configuration. Seventeen rotations yielded 
a highly satisfactory simple structure in which no more rotations were 
evident. On June first the code was obtained and opened for the first 
time. No further rotations were made. The structure obtained by ro- 
tating “blindly” yielded the same psychologically meaningful factors 
as those obtained in numerous other studies in which the rotations 
were made with a knowledge of the names of the test vectors. 

The factorial matrix obtained after rotation is presented in Table 


TABLE 6 


Direction Cosines of the Reference Vectors* 








N wy. 2s 2a FR Se ae Oe 


Iienati@ggyumeP K&B RBHan @ B&B & 

II -52 28 11 O01 -83 25 00 44 -30 -05 
III 35 33 28 -23 24 28 -15 -40 -46 -01 
IV 386 15 -48 16 -29 -47 -04 24 -40 29 
V 48 -45 11 36 -49 35 35 -15 -20 -20 
VI 12 57 -34 -26 -41 32 -06 -39 62 -09 
VII 16 -28 -58 -38 33 34 -03 46 00 -39 
VIII 29 -26 39 -72 -44 -25 -08 24 14 02 
IX 01 -25 14 22 -11 17 -90 -15 18 09 
X -34 -26 -11 -10 06 46 13 -26 -20 82 














* All entries have been multiplied by 100 to eliminate the decimal. 


TABLE 7 


Cosines of the Angles Between the Reference Vectors* 


N y Ss M - D I a ae ae 


99 

-08 101 

-03 -11 100 

-12 -08 -02 101 

-17 01 -10 00 86100 

-16 -O07 -04 -04 07 102 

02 -01 ~-10 04 -02 01 100 

-12 -20 -19 -16 00 =-31 11 8699 

—02 15 -08 -22 ~-12 03 -21 -17 99 
-27 -04 04 08 -01 02 -02 -25 -24 99 











QnbaeSOvEndS 





* All entries have been multiplied by 100 to eliminate the decimal. 
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5, and the transformation matrix (A) leading from the centroid ma- 
trix (F’.) to the final rotated matrix (V) by the equation 
V=F.A 


is given in Table 6. 
The correlations between the reference vectors, obtained by the 
matrix multiplication (A’ A), are presented in Table 7. 
TABLE 8 


Direction Cosines of the Primary Vectors* 











IX 10 -17 06 27 -09 20 -87 08 10 10 
X -13 -25 -17 -21 00 29 138 -10 -05 61 


* All entries have been multiplied by 100 to eliminate the decimal. 


The correlations between the primary vectors, obtained by the 
equation 





R,,~=H H', 
are shown in Table 9. 
TABLE 9 


Correlations Between the Primary Vectors* 


N . a So es @ & £2 2 € 








100 
24 100 
21 21 100 


28 19 17 100 

24 05 16 10 100 

88 24 20 22 03 100 

00 -02 10 00 06 -03 100 

44 35 34 86 14 46 -04 100 

27 O08 22 82 21 15 21 87 100 

44 18 17 18 15 23 05 46 39 100 


* All entries have been multiplied by 100 to eliminate the decimal. 


Qa SvEnde 
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The matrix H’ is given by the equation 
H’= (A' A)*D, 
where D is a diagonal matrix the entries in which are the normalizing 


constants of the corresponding columns of A. The columns of this 
matrix (H’) contain the direction cosines of the primary vectors 


(see Table 8). 


TABLE 10 


Correlations of the Tests with the Primaries* 








7, "2 2 2 2 TT 2 2 <S 


91 23 28 22 22 21 -04 38 27 45 
94 30 20 30 20 40 02 48 26 45 
91 26 18 30 21 40 00 48 28 48 
49 33 34 80 25 27 16 62 34 62 
. 85 238 62 34 56 
34 16 32 29 11 20 41 58 42 ~— 58 
42 31 33 37 14 49 -06 95 38 46 
44 34 32 38 138 44 -07 96 37 44 





OCWONAH MNP WDe 
~~ 
ary 
oo 
~ 
bo 
© 
bo 
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° 
hes 


34 20 28 31 18 26 42 -04 88 26 19 





* All entries have been multiplied: by 100 to eliminate the decimal.: : 
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Applying the transformation (H’) to the centroid matrix in the 
manner 


R;,=—F.H, 


the correlations of the tests with the primary vectors are obtained 
(see Table 10). These correlations are the validity coefficients of the 
tests as measures of the primaries. 


III. Interpretation and Discussion 


The most interesting part of a factor analysis is the psychologi- 
TABLE 11 
Simplified Rotated Factorial Matrix* 
N V Ss ee F D I A B C 


21 45 32 
24 64 21 31 


20 41 22 
26 71 23 


19 50 28 
28 55 


11 (19) 65 


10 46 24 
32 57 


34 (19) 30 

6 387 626 25 
30 29 30 24 

31 20 20 25 26 


29 28 22 30 
5 23 027 25 


138 22 31 35 
4 24 80 


* All entries have been multiplied by 100 to eliminate the decimal. 
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cal interpretation of the factors obtained. To facilitate the interpre- 
tation we have reproduced the rotated matrix (Table 11), excluding 
all projections under .20 and rearranging the rows to make the struc- 
ture more apparent. 

The number factor.—Considering first the number factor, N, it 
may be seen that Two-Digit Addition has the highest projection, with 
Three-Digit Addition next, followed by Four-Digit Addition. The or- 
der of these three tests in their projection on number indicates that 
the simpler tests are better tests of this factor. This is borne out by 


(1) Two-Digit Addition - --- .74 (22) Multiplication - - - - - - - 64 
(2) Three-Digit Addition - - - .72 (21) Addition - -------- 45 
(3) Four-Digit Addition - --- .66 (18) Identical Numbers - - - - .22 


the relation of Multiplication and Addition, the former being higher 
than the latter. The multiplication problems are the product of two 
digits by one digit, whereas the addition problems are the sum of six 
two-digit numbers. These results are strong evidence against the hy- 
pothesis that number ability is in the nature of a serial response pro- 
cess. The only other test having a projection greater than .20 is Iden- 
tical Numbers, with a projection of .22. These results on the number 
factor indicate that it has a speed characteristic. 

The rotated factorial matrix does not yield any information on 
the various hypotheses that the battery was designed to investigate 
because the primaries are oblique. However, information on these 
points may be obtained from the correlation matrix (Table 4) and 
from the validity coefficients of the tests (Table 10) as measures of 
the primary. These data are assembled in the following four tables. 

The three alphabet tests were designed to check the hypothesis 


TABLE 12 


Correlations of Alphabet Tests with Number Tests 








Two-Digit Three-Digit Four-Digit Multipli- Addition 
Addition Addition Addition cation 





(7) Alphabet I - - - 363 449 476 345 266 
(8) Alphabet II - - 383 472 489 355 283 
(9) Alphabet III - - 399 493 507 337 301 





that practice with a set of rules improves a test as a measure of num- 
ber ability. If this is correct, the correlations of tests Alphabet I, 
Alphabet II, and Alphabet III with the number tests should progres- 
sively increase. This block of correlation coefficients is presented 
above in Table 12. 
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TABLE 13 
Validity Coefficients of Alphabet Tests with the Number Primary 








(7) Alphabet I --------- 42 
(8) Alphabet II --------- 44 
(9) Alphabet III -------- 46 





The correlations all progressively increase with the one exception of 
the correlation between Alphabet III and Multiplication. Similarly, 
the validity coefficients of the three alphabet tests as measures of num- 
ber ability show a slight increase as the subjects become more prac- 
ticed with the set of rules. It is not to be expected that the amount of 
practice obtained by the subjects while taking these alphabet tests 
will yield a familiarity with the system in any way comparable with 
their familiarity with the number system. 

The AB, ABC, and Forms tests were designed to throw light on 
the hypothesis that the more familiar or well-established the symbol- 
ism the better the test is as a measure of number. If this hypothesis 
is correct, the correlations of the AB and ABC tests with those for 
number ability should be distinctly higher than the correlation of the 
Forms test with the number tests. Table 14 shows that the predic- 


TABLE 14 
Correlations of AB, ABC, and Forms Tests with Addition and Multiplication Tests 








Two-Digit Three-Digit Four-Digit Multipli- Addition 
Addition Addition Addition cation 





(4) AB- ------ 488 502 515 408 396 
(5) ABC - ----- 407 455 470 358 371 
(6) Forms - - - - - 301 385 396 284 355 








tions of the hypothesis are borne out without exception. The validity 
coefficients of these three tests as measures of number ability also 
support this conclusion. 








TABLE 15 
Validity Coefficients of AB, ABC, and Forms Tests with the Number Primary 
(4) AB - ------------ 49 
(5) ABC - ----------- Al 
(6) Forms - ---------- 84 





The consistent difference between the AB and the ABC test in 
the foregoing data also substantiate the conclusion that the simpler 
or less involved the manipulations called for in the test, the better is 
the test as a measure of number ability. 

It is almost impossible to prove conclusively the original hypo- 
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thesis that number ability represents the rapidity with which well- 
established associations may be recalled and manipulated. The only 
conclusion which may safely be drawn is that the hypothesis is not 
disproved. All the data bearing on deductions from the hypotheses 
are in agreement with it. 

The verbal factor.—The second factor, V, has the following four 
tests high on it: 


(23) Completion - ------- 66 
(24) Same-Opposite - - - - - - - 64 
(15) Size Comparison - - - - - - .49 
(20) Verbal Enumeration---- .41 


There are no other tests with projections higher than .20. This factor 
is obviously the verbal factor. Completion and Same-Opposite are the 
two tests designed to isolate the verbal factor. Verbal Enumeration 
has been found previously to have a projection on the verbal factor. 
The fact that the new test, Size Comparison, has a projection on this 
factor is in agreement with the interpretation of the factor as verbal 
ability. 

The space factor.—The third factor, S, is defined by the follow- 
ing tests: 


(26) Figures - --------- Bs | 
(25) Cards - ---------- 65 
(19) Identical Forms - - - - - - 50 
(31) Number Patterns - - - - - .20 


This factor is readily seen to be the space factor, inasmuch as Figures 
and Cards are the two tests for this primary. Identical Forms has 
also been found previously to have a projection on the space factor. It 
is interesting to observe that none of the three tests are pure, the first 
two having projections on deduction and the third having a projection 
on perceptual speed. This indicates that we do not yet have clear in- 
sight into the psychological nature of this factor, or perhaps that the 
freedom given the subjects in adopting a work method results in some 
subjects using deductive ability to do the tasks in the Figures and 
Cards tests. 

Memory.—The fourth factor, M, is obviously the Memory factor 
with no other tests high than the two included for the purpose of iso- 
lating this ability. Although this factor is called memory ability, it 


(28) Word-Number - - - - - - - 55 
> SN. 3S ewes 53 


should be more strictly defined as rote learning, as both Word-Number 
and Initials are tests of rote learning with immediate recall. The 
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score is more representative of a point on the ascending learning 
curve than of a point on the descending forgetting curve. 

Perceptual Speed.The fifth factor, P, we have identified as per- 
ceptual speed. The tests with highest projections on this factor are 
the three designed for the purpose of clarifying the definition of per- 
ceptual speed. 


(11) Scattered ~’s (rowed) - - .65 (19) Identical Forms - - - - - .28 
(12) Seattered x’s (pied) - - - .55 (29) Letter Grouping - - - - - 28 
(10) Digit Cancellation ---- .46 (20) Verbal Enumeration - - - .22 
(18) Identical Numbers ---- .81 (31) Number Patterns - - - - - .20 


Certain suggestions are contained in these results which would 
be worth further investigation. There were considerably more can- 
cellations per unit time in the Digit Cancellation test than in the Scat- 
tered X’s (rowed) test, so the indications are that the more scanning 
and the less cancelling called for by a test the better it is as a test of 
perceptual speed. The Scattered X’s (pied) test would probably be a 
better test of perceptual speed if administered in another manner. The 
subjects were instructed that there were only seven x’s on a page and 
that they might go on to the next page whenever they wanted to. It 
would probably improve the test if they were required to get all the 
x’s on each page before going on to the next, or perhaps, even better, 
if they were given a time limit of about twenty seconds on each page. 
An inspection of the fifth column of Table 11 indicates that all the 
tests with projections of .20 or greater involve the rapid scanning 
of the test content. 

The Deductive Factor.—The sixth factor, D, may be identified as 
the deductive factor. The three tests included in the battery for the 
identification of this factor (Arithmetic, Number Series, Mechanical 
Movements) have their highest projections on it. 


(32) Arithmetic - - ------ 57 (30) Marks - --------- .29 
(33) Number Series - - - - - - .56 (31) Number Patterns ---- .25 
(25) Cards - --------- 36 (26) Figures - -------- .23 
(84) Mechanical Movements -_ .30 


Two of the space tests have significant projections on this factor 
for a possible reason already indicated. Two other tests, Marks and 
Number Patterns, have projections of .29 and .25 respectively on this 
factor. This is in agreement with the interpretation of this factor as 
deduction. It is interesting to note that with the definition of number 
ability by new tests with a greater speed characteristic than those in 
former studies, Arithmetic and Number Series have lower projections 
on the number primary.* 


* Dr. H. O. Gulliksen has pointed out that these deductive tests seem to have 
&@ serial response character. 
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The inductive factor.—Those tests which have their highest pro- 
jection on the seventh factor, J, are: 


(6) Forms - --------- 37 (31) Number Patterns ---- .26 
(80) Marks - --------- 30 (5) ABC - ---------- 23 
(21) Addition - -------- 32 (29) Letter Grouping - - - - - -22 


Addition, which is primarily a number test, has a projection of 
.382 on this factor. Letter grouping and ABC also have projections 
greater than .20. Because of the small amount of variance accounted 
for by this factor, it is both difficult and risky to interpret. Jnasmuch 
as Marks, Number Patterns, and Letter Grouping were included in 
the battery to identify the inductive plane, this factor might very 
tentatively be called induction. 

Other factors.—The eighth factor, A, is primarily identified by 
the triplet of alphabet tests, a number of other tests having projec- 
tions between .20 and .30. A factor defined by a triplet of tests so 
similar in nature as the three alphabet tests cannot be interpreted 
with any certainty. For that reason no attempt will be made here to 
indicate the psychological significance of this factor. Similarly, the 
ninth factor, B, is identified by the three substitution tests, with no 
other test having a projection on it as high as .20. This factor is an- 
other defined primarily by a triplet, and no attempt will be made to 
identify it. 

The last factor, C, has very low variance and may be considered 


to be a residual factor. 


Discussion 


The reasons why such tests as AB, ABC, Forms, and the three 
alphabet tests have no projection on the number factor is that most 
of their variance is explained by other common factors, in this case 
primarily factors A and C, which have no “pure” tests to identify 
them. The result is that by rotating strictly to configuration, the num- 
ber plane is passed through these tests, causing the configuration to 
be oblique. Hence, as may be seen from Table 9, the number primary 
correlated .44 with each of these two primaries, A and C. It is these 
correlations which have absorbed the potential projections of these 
tests on the number factor. It is assumed in this argument that the 
tests for the number factor do not call upon any non-number factor 
present in the alphabet tests. An alternative assumption is possible. 
It may be assumed that the alphabet tests are “pure” tests involving 
no number ability, but that the number tests are complex in that they 
involve not only the number factor but also the common factor de- 
fined by the alphabet tests. However, it seems more reasonable that 








CLYDE H. COOMBS 187 


the simple number tests involve only one of the two factors and the 
more complex alphabet tests involve both of them. 

Indications that word tests have significance to the number fac- 
tor are apparent in several studies already made. One of the tests 
described by Thurstone in his monograph, Primary Mental Abilities,* 
is Free Writing, the score on which is the total word count of a theme 
the subject is asked to write. This test had a projection of .295 on the 
number factor. In Thurstone’s recent monograph, Factorial Studies of 
Intelligence,; the four highest tests on the word factor are Prefixes, 
First Letters, First and Last Letters, and Suffixes. 

In the test Prefixes, the subjects were instructed to write all the 
words they could that began with “con.” In First Letters the subjects 
were instructed to write all the words they could that began with the 
letter “‘s.” In First and Last Letters the subjects were instructed to 
write all the words they could that began with “t’” and ended with 
“e.” In Suffixes they were instruction to write all the words they 
could that ended with “tion.” 

None of these tests had a projection on the number factor be- 
cause in rotating strictly to configuration the number plane was passed 
through these tests. But, as in the case of the present study, the re- 
sult was oblique primaries and the correlation between the axes ab- 
sorbed the potential projections of these tests on number. The corre- 
lation between the number and word primaries in this battery was .33. 

In another study, as yet unpublished, the three tests for the word 
factor were First Letters, Four-letter Words, and Suffixes. The num- 
ber factor and word factor were correlated .47 in this study. 

By refinement of these word tests it should be possible to devise 
word tests which do not involve number ability and, as opposed to 
these, others in which the content is verbal but which do not involve 
the word factor. It is not certain whether the manipulatory character 
or the associative character of these word tests is the basis of their 
relation to number. A type of word test for the investigation of this 
problem may be suggested here. The subject could be given a list of 
words which were the first members of pairs of words commonly asso- 
ciated, such as cup-saucer, man-woman, knife-fork. He could then be 
required merely to write the first letter of the associated word. Such 
a test would involve a minimum of manipulatory processes. On the 
other hand, a test like Free Writing, described above, probably has 
more of a manipulatory than an associative character. 


*L. L. Thurstone. Primary mental abilities, Psychometric Monographs, 1938, 
No. 1. Chicago: Univ. Chicago Press, pp. 121. 

+L. L. Thurstone. Factorial Studies of Intelligence, Psychometric Mono- 
graphs, 1941, No. 2. Chicago: Univ. Chicago Press, pp. 94. 
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With regard to the interpretation of the factors in general, it 
should be pointed out that the identification of one of the factors as 
perceptual speed does not imply that it is the speed ability. Certain 
aspects of the classical problem of speed versus power may be reinter- 
preted in the light of the results of multiple factor analysis. There 
does not appear to be a single speed ability, but a number of them. If 
a test is composed of items of equal difficulty and, though not neces- 
sarily, of items of a low level of difficulty, then the scores on that test 
represent the amount done in a given time and measure a speed ability. 
In this sense at least three of the factors obtained in this study, num- 
ber, space, and perceptual speed, are speed abilities. 


VI 


Summary and Conclusions 


A battery of thirty-four tests was given to 223 high school sen- 
iors in the city of Chicago. The battery included sixteen tests for the 
identification of seven of the primary mental abilities and eighteen 
experimental tests for the investigation of particular hypotheses. The 
study was undertaken primarily to investigate the hypothesis that 
number ability represents the agility with which an individual can 
manipulate a symbolic system according to a specified set of rules, 
with two restrictions: that the symbolic system be familiar, and that 
the set of rules be highly practiced. Certain deductions were made 
from this hypothesis and non-numerical tests were designed to check 
these deductions. A secondary problem was to obtain a clearer iden- 
tification of the perceptual speed factor in order that its relation to 
number ability might be determined. 

The ten factors of the centroid matrix were rotated to simple 
structure without knowledge of the identity of the tests. Seven of 
the ten factors were given a psychological interpretation, two were 
triplets and non-interpretable, and one was a residual factor. The 
seven primaries that were given psychological interpretation were 
number, verbal, space, memory, perceptual speed, deduction, and in- 
duction. 

Conclusions.—1) The number factor is most clearly identified by 
very simple number tests. 

2) The perceptual speed factor is most clearly identified by can- 
cellation tests, and the more scanning that is involved and the less can- 
celling the better the test is as a measure of perceptual speed. 

3) A test involving manipulation of a symbolic system is a bet- 
ter measure of number ability the more familiar the symbolism: 
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4) A test involving operations according to a set of rules be- 
comes a better measure of number ability with practice. 

5) The data are in disagreement with the hypothesis that num- 
ber ability is in the nature of a serial response. 

6) The results are in agreement with the hypothesis that num- 
ber ability is characterized by a facility in manipulating a symbolic 
system according to a specified set of rules. 
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THE EVALUATION OF DETERMINANTS 


P. S. DWYER 
UNIVERSITY OF MICHIGAN 


The numerical evaluation of determinants with a modern com- 
puting machine is discussed. Various methods are presented and 
their relations to each other are indicated. The methods presented 
parallel those developed in the previous paper on “The Solution of 
Simultaneous Equations.” Especially emphasized are the Abbreviat- 
ed Doolittle and the Compact methods. Additional topics include the 
evaluation of partially symmetric determinants 4 means of symmet- 
ric methods and the evaluation of determinantal ratios. 


Introduction 


Determinants are very useful in discovering the theoretical prop- 
erties of the solutions of simultaneous equations, but they have not 
been found very useful in obtaining the numerical solutions. This is 
particularly true in least squares and correlation theory where ap- 
proximate solutions only are demanded and where one usually has ac- 
cess to modern computing machines. Thus authors of text books on 
statistics frequently recommend non-determinantal methods for the 
numerical solution of normal equations. See, for example, (1, p. 67) 
(2, p. 36) (3, pp. 119-124). In a pervious article (4) the writer has 
indicated a number of these solutions. 

It is possible to apply these methods to the evaluation of determi- 
nants. It is the purpose of this paper to show how this can be done. 
The reader who is familiar with the earlier paper should have little 
trouble in understanding the development even though the present 
paper is somewhat more condensed than the earlier one. 

For purposes of brevity we use the fourth-order determinant 


Ay, My, Ag ay 
Ayo Ase Azo Ase (1) 
Qy3  Ae3 Asg Anz 
Ayg Ang Age Wag 


with the implication that the methods outlined are to be extended to 
the n’th order. As an illustration we take the determinant 
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ae 2 6 2 | 
pel 410 8 4) (2) 
3. 2 2 
: & * 


in which the elements are exact. It is evident from the definition of a 
determinant that the exact value of A is expressible in terms of four 
decimal places. This value is A = .3660. In working with the different 
methods we attempt to carry the approximations to at least four 
places. 

The determinant above is symmetric. However, it can be used to 
illustrate a non-symmetric determinant by assuming a;; # a@;;. The 
reader will note that this determinant (2) is the determinant of the 
coefficients of the illustration of the paper on the solution of equations 
(4). 

In all the methods which follow it is our plan to eliminate the first 
column, the second column, etc., in order. It is agreed that such an 
order is arbitrary. However, the technique is sufficiently general since 
if the rows and columns have the same order, they may be inter- 
changed in all possible ways. 


First Method. Method of Division 


In this method, the elements of the first row of A are divided by 
a,, , the elements of the second row by az,, etc. The first row of the 
resulting determinant is then multiplied by —1 and added to the sec- 
ond row, to the third row, and to the fourth row, in turn. We have 
then 


| Q'o9.. Oso. go. 
eo , a4 
A = Gy ie Ais Are | Mos. Ass. Dasa |, (3) 
| | 
| Dear Dar Wasa | 


: Q;; a; J : Pen 
where @’j;.. = = — — . If we treat the resulting determinant simi- 
1j 11 


larly, we get 
; : | @s312  O'43.22 | 
A = Ayy Dye Uys My4 W'22.1 W234 Wear, 2 | ’ (4) 
| Dsar2  Uasere 
where 
Wija Vier | 
’ 








’ — 
0 jj.22 = 


’ 
2j-1 Q 20-1 
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and finally 
A = Gy, Aye Myg My 4 W' 99.1 W'23.1 W' 24.1 W' 33.12 W'g4-12 V'44.228 (5) 


is obtained by multiplying the entries in (5) or, if one prefers, by 
evaluating the determinant of (4) and multiplying by the indicated 


values. 
The method is illustrated i in Table 1 where 


A= = (1.0) (.4) (.5) (.6) (2.1000) (.2000) (.2667) { (7. 3810) (3. 8091) 
— (—.7440) (—1.1905)} = .3660. 


TABLE 1 
METHOD OF DIVISION 
1.0 A 5 6 
4 1.0 3 4 
5 a] 1.0 Z 
6 A 2 1.0 


1.0000 .4000 .5000 .6000 
1.0000 2.5000 .7500 1.0000 
1.0000 .6000 2.0000 .4000 
1.0000 .6667 .3333 1.6667 
2.1000 .2500 .4000 

2000 1.5000 -.2000 

2667 -.1667 1.0667 

1.0000 .1190 .1905 
1.0000 7.5000 -.1000 
1.0000 -.6250 3.9996 
7.3810 -1.1905 

-.7440 3.8091 

1.0000 -.1613 

1.0000 —5.1198 

—4.9585 

3660 


| This method is the least satisfactory of the various methods pre- 

sented. The symmetry of the original determinant is lost with the 
first set of divisions. A large number of divisions is necessary while 
the method demands n(n + 1) rows (though but n(n + 1) — 3 rows 
are used if the second-order determinant is evaluated directly). 


Second Method. Method of Single Division 


In the method of single division, the reduction is accomplished by 
a division of the first row by its leading element. Thus 
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ber Bsr 
Giz Ges Ass Ms 


| Gre Gag Ase ae 




































where bj, = a . We multiply the first row by —4a. and add to the 


1 


second, by —a,; and add to the third, by —a,, and add to the fourth 
and get. 

Gee. Ase. Mar | 
, (7) 


A = yz | Moz. Aggy Oggaa 


Ae4.1 Az4.1 Qa4-1 








where 4;;.. = a; — a = . Similarly, we eliminate the first row of 
(7) and get 
1+)" ™* » (8) 
ein ie | 
where Qi j.12 = Qija — . We thus get 
2-1 
A = yy Ao.1 O33.12 Vas.123 » (9) 


The method is illustrated in Table 2(a). To the first four rows of 4 
are added the division row. The values in the next grouping are then 
obtained by subtracting from the element the product of the row head- 
ing and the columnar base. The division is made and entered, the 
next group of rows computed, etc. 


A= (1.0) (.84) (.7381) (.5903) = .3660. 


A somewhat shorter variation utilizes the conventional method 
of evaluating a second order determinant. Thus 


A= (1.0) (.84) [(.7381) (.6095) — (—.1190)?] = .3660. 


This method is an approximation method since divisions are in- 
troduced. It maintains the symmetry property since 4a;;.. = aj;.. It 


demands i — or [ a 


5 — 2)] rows. The method is, essen- 














P. S. DWYER 195 


TABLE 2 


METHOD OF SINGLE DIVISION 




















(a) () (c) 
10 .4 5 6 10 4 5 6 10 4 5 6 
41.0 3 4 — 1.0 3 4 410 3 a 
5 38 1.0 2 -_-— 1.0 2 5 8 1.0 2 
6 A 2 1.0 _--— _ 1.0 6 4 2 1.0 
10 4 5 6 1.0 .4 5 6 10 .4 5 6 
84 10 16 84 10 16 84 10 16 
10 75 -10 _ 75 -.10 10 _ _ 
16 8 -.10 64 —_ _ 64 16 _ _ 
1.0000 .1190 .1905 1.0000 .1190 .1905 1.0000 .1190 .1905 
-7381 -.1190 -7381 -.1190 -7381 -.1190 
-.1190 .6095 —  .6095 -1199 — 
1.0000 —.1612 1.0000 -.1612 1.0000 -.1612 
5903 5903 -5903 
3660 -3660 3660 





tially, that of Chio (5). If a leading element is 0, some other order 
must be chosen. 


Third Method. Method of Single Division-Symmetric 


In case the determinant is symmetric, it is not necessary to record 
the values below the main diagonal. The proper entry is found by sub- 
tracting from aj; the product of b;, and a;,. The bj, term is at the 
bottom of the column while the a;, term is obtained by moving to the 
left to the main diagonal and taking the columnar heading. 

The illustration is given in Table 2(b). 


Fourth Method. Abbreviated Method of Single Division 


In the abbreviated method of single division, the first row and 


the first column only of each new grouping is computed. The proper 
entry is finally obtained by subtracting one product out of each group- 
ing. Thus 


Qus.10 = (.2) — (.5) (.6) — (.10) (.1905) = —.1190. 
See Table 2(c) for illustration. 


Fifth Method. Abbreviated Method of Single Division-Symmetric 
The columnar entries of the fourth method may be eliminated if 
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the determinant is symmetric, since the top rows give the multipliers. 
Thus 

43-12 = (.2) — (.6) (.5) — (.10) (.1905) = —.1190 

Oger03 = 1.0 — (.6)2 — (.16) (.1905) — (.1190) (—.1612) = .5908 . 
See Table 3(a). 


TABLE 8 
ABBREVIATED DOOLITTLE METHOD 





























(a) | (b) 
10 4 5 6 | a 2 5 6 
— "| 3 oo ee 3 4 
—— 10 2); — 1.0 2 
ink. “tee iia 1.0 is ne 1.0 
10 4 5 6 10 4 5 
84 10 16 10 4 5 6 
1.0000 .1190 .1905 84 10 16 
7381 -.1190 1.0000 .1190  .1905 
1.0000 -.1612 7381 -.1190 
5903 | 1.0000 -.1612 








.3660 | 5908 
| 3660 


| 
| 





Sixth Method. Abbreviated Doolittle Determinantal Method 


This method is essentially the same as the last method. The first 
row is repeated at the first grouping so that the technique is simply 
to subtract the products of elements in paired rows from the first ele- 
ment. This technique is easily carried out once it is understood. The 
final evaluation of the determinant, it is remembered, is 


A= (1.0) (.84) (.7381) (.5903) = .3660. 


The Abbreviated Doolittle method can be expanded to give the con- 
ventional Doolittle method. It appears that Horst (6) was first to 
evaluate determinants with the use of the Doolittle method. He was 
chiefly interested in the correlation determinant but his method can 
be applied more generally. He derived the basic formula for evaluat- 
ing determinants by any of the variations of the method of single di- 
vision. 
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The equivalent of the formula (9) has been used more recently 
by Reiersol (7) in computing all the principal minors of a determinant. 
His notation differs somewhat from the present notation but his tech- 
nique results from an application of the methods and formulas of the 
method of single division. 


Seventh Method. The Method of Multiplication and Subtraction . 


It was noticed in method one that the work was somewhat abbre- 
viated by the use of (4) rather than (5). This leads to the suggestion 
as to the evaluation of the whole determinant by ab — cd methods. 


This can be done. 
The elements of the 2nd, 3rd, and 4th rows of A are multiplied by 


i. . , 
@4,, and a is placed outside the determinant to compensate. The first 
11 


row is then multiplied by —a,2. and added to the second, by —a,; and 
added to the third, by —a,, and added to the fourth. In the resulting 
determinant we have 





| Gy, May As Qs, 
A= : | 0 Aso. Aves ees : (10) 
ss 7 | 0 Ass. Asc Pore 
H 
| 0 Ag. Asa. Aes 





where A jj.. = Qi; G1, — Aj, A;. 








Hence 
| Aso Age. Ader 
1 
A= = Ags, Agsa Aas (11) 
O41 
Aga Asar Au | 








Continuing this process we get finally 


Som Fy 1 
A= (G-) ( x) Whine. (12) 


The successive steps in the computation are shown in Table 4(a). 
This method has certain advantages. No divisions are needed for 

the forward solution so that the computation is eased. Also the value 

A= aa a (.30744) is exact, and the solution 4 = .3660 is ex- 

act. Of course, exact values cannot be obtained beyond machine ca- 

n(n + 1) 
2 





pacity. The method needs + 1 rows. 
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TABLE 4 


METHOD OF MULTIPLICATION AND SUBTRACTION 




















(a) (5) (c) 
10 4 = 6 6 10 4 6 6 10 4 6 6 
410 2 4 —10 3 A 410 3 A 
5 38 1.0 2 — —10 2 5 28 1.0 2 
6 4 2 1.0 -_-- 1.0 6 4 2 1.0 
84 .10 16 84 .10 16 84 .10 16 
10 = .75 -:10 — .15 -.10 10 — 
16 -.10 64 _— 64 16 — —_ 
6200 -.1000 -6200 -.1000 -6200 -.1000 
-.1000 .5120 — 5120 -.1000 _ 
80744 30744 30744 
3660 3660 3660 








Eighth Method. Method of Multiplication and Subtraction-Symmetric 


In case the determinant is symmetric, the entries below the main 
diagonal may be left blank and the top entries used for computation 
as in method three. The illustration is given in Table 4(b). 


Ninth Method. Abbreviated Method of Multiplication and Subtraction 


The method can be abbreviated, as in method four, by recording 
only the entries of the first column and the first row in each matrix. 
The illustration is presented in Table 4(c). For example, the value of 
Aganes iS : 


Aui123 = {[1.0000 — (.6000)?] .8400 
— (.1600)?} .6200 — (—.1000)? = .80744. 


Tenth Method. Abbreviated Method of Multiplication and 
Subtraction-Symmetric 


In case symmetry is present it is possible to eliminate many of 
the rows of the last method. As a matter of fact, but 2” rows are 


needed to indicate the solution. The statement of the problem requires 
n rows. The next row gives the entries a2;.. (or @j2.), the next @sj.2 
(or @iz..2), etc, The computational work is shown in Table 5(a@). For 
example 

A4s.12 = [ (.2000) (1.0000) 


— (.5000) (.6000) ] .8400 — (.1000) (.1600) = —.1000. 
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A somewhat better form, though it requires an additional row, is 
indicated in Table 5(b). This differs from Table 5(@) only in the 
repetition of the first row at the end of the first » rows. It is then 
necessary only to take an element a;; , multiply it by the element at the 
left of the (n + 1)’st row and subtract the product of the 7’th and j’th 
columnar entries in this row, multiply by the leading entry of the next 
row, etc. Thus 


44-123 = {[ (1.0000) (1.0000) — (.6000)?] .8400 
— (.1600)?} .6200 — (—.1000)? = .30744, 
and 


1 1 


= 17,0000) 8400 (30744) = 3660. 


A 





TABLE 5 


CoMPACT METHOD 








(a) (b) 











1.0000 .4000 .5000 .6000 1.0000 .4000 .5000 .6000 
— 1.0000 .3000 .4000 — 1.0000 .3000 .4000 
— — 1.0000 .2000 _ — 1.0000 .2000 
_ — — 1.0000 —_ _— 1.0000 

8400 .1000 .1600 1.0000 .4000 .5000 .6000 
-6200 -.1000 8400 .1000 .1600 
80744 6200 -.1000 
3660 80744 
3660 








This method is recommended as a most compact method in ob- 
taining numerical approximations to the values of determinants of 
high order. 

In all these methods a check column can be used. 


The Evaluation of Partially Symmetric Determinants by 
Symmetric Methods 


We may define a partially symmetric determinant to be one in 
which there is a symmetric minor, of order two or more, of any ele- 
ment of the principal diagonal. If this minor is of order one less than 
the order of the determinant, we may call the determinant “almost 
symmetric.” Thus the determinant 
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| 
| Ga Qe, Ajy 
A=} Aye Agg Ajo 





G13 A23 Ajs | 


is “almost symmetric” if @,. = a2, since the minor of aj; is then sym- 
metric. 

We wish to use the symmetric methods in the evaluation of 4. 
This is done by inserting columns, for computational use, to make the 
first n rows and column symmetric. Thus we might evaluate A by the 
use of 

Ay, a1 gy Hj 


Qjo Aoe Aso Aj2, With a=, 
Giz G23 — Ajg 
and the solution is indicated by 
Azo.1 A32.1 Ajen 
— A js.12 ’ 


Ajs.12 


with A= ~ i” 


If symmetry had been lacking in two columns, it would have been nec- 
essary to introduce two computational columns, ete. 
This device can be adapted to the Abbreviated Doolittle method 


similarly. 


The Simultaneous Evaluation of Almost Symmetric Determinants 


It is desired to evaluate a number of determinants which are 
alike aside from the last column. If the number of these determinants 
is small, it is advised to use the method above and to add additional 
columns. Thus the evaluation of 


mm A SS Ge | 
As 410 8 Gs, | “a 
| 2B. BAD ee. 

| 
ys: A. CS] OY 
is outlined in Table 6 by the compact method for different values of 
Aj; 
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TABLE 6 


ALMOosT SYMMETRIC DETERMINANTS 

















check 
10 4 5 6 2 8 3.5 
410 3 4 A 6 3.1 
5 .8 1.0 2 6 4 3.0 
S “22 1.0 8 2 3.2 
LO <2 6 6 & 8 8.5 
.84 .10 16 382 .28 .170 

.6200 -.1000 .38880 -.0280 -8800 

80744 .86120 -.17640 49224 

A=.8660 .4800 -.2100 .5860 





When aj; = .6, @j. = .4, aj, = .2, @j4 = 1.0, then 4= 3660. 

When aj, = .2, aj. = .4, aj; = 6, aj,—= .8,then4= _ .4300. 

When aj, = .8, @j2 = .6, aj3 = .4, aj, = .2, then A = —.2100. 
From the check column, when aj; = 3.5, @j2 = 3.1, @js = 3.0, aj, = 3.2, 
then A = .5860. 

If large numbers of these determinants are desired, however, it 
is preferable to divide the last column into its aj; , @j2 , Gs , Aj, , COM- 


ponents, leaving a column for each component. The specified values 
a;; can be inserted in the result. The form is shown in Table 7 where 


TABLE 7 (a) 


SIMULTANEOUS DETERMINANTS—ABBREVIATED DOOLITTLE METHOD 

















a5, jo ais is check 
1.0 5 6 1 0 0 0 3.5 
— 1.0 3 m1 0 1 0 0 3.1 
-_-_ -— 1.0 2 0 0 1 0 3.0 
-_-_ — — 1.0 0 0 0 1 3.2 
10 «44 5 6 1 0 0 0 3.5 
10 «4 5 6 1 0 0 0 3.5 

84 -10 16 -40 100 0 0 1.70 
1.0000 .1190 .1905 -.4762 1.1905 0 0 2.0238 





-7381 -.1190 -.4524 -.1190 1.0000 .0000 1.0476 
1.0000 -.1612 -.6129 -.1612 1.3548 .0000 1.4193 


5903 -.5966 -.2097 .1612 1.0000 .9451 
38660 -.3699 -.1300 .1000 .6200 .5860 
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the fourth column is inserted to make possible symmetriic methods. 
The Abbreviated Doolittle method is shown in Table 7(a), while the 
“compact” method is presented in Table 7(b). 


TABLE 7 (6) 


SIMULTANEOUS DETERMINANTS—COMPACT 














a;, Ajo a5, ai, check 
10 64 5 6 1 0 0 0 3.5 
— 10 3 A 0 1 0 0 3.1 
_-_ — 1.0 2 0 0 1 0 3.0 
-_-_ — — 1.0 0 0 0 1 3.2 
10 «(4 5 6 1 0 0 0 3.5 

84 10 16 -40 100 £0 0 1.70 
6200 -.1000 -.38  -.10 84 0 8800 


30744 -.3108 -.1092 .0840 .5208  .49224 
.3660 -.3700 -.1800 .1000 .6200 .5860 








It follows that the value of A is 
A = —.3700a;, —.1300a;. + .1000a;, +.6200a;, . 


The values of 4, for the value of a;; used in Table 6, and for other 
values of a;; , are given in Table 8. 


TABLE 8 








‘=i s2 ‘=3 4=-4 A 











Aj, -.3700 -.1300 .1000 .6200 ~-.2100 
a;; 6 A 2 1.0 .3660 
5; 2 A 6 8 .4300 
a; 8 6 A 2 -.2100 
a;, 3.5 3.1 3.0 3.2 .5860 
5; 5 3 1.0 2 .0000 
a;; A 1.0 3 A .0000 
a;, 1.0 A 5 6 -0000 
a;; 1.0 1.0 1.0 1.0 .2200 
etc. 





In general 4 = 5 a;; A;;, where the A;; are determined. It is to be 
noted that the quantities A;; are the co-factors of a;; and hence that 
the method is a method of determining the co-factors of the elements 
of a given column. 
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The Evaluation of Determinantal Ratios 


In the solution of equations it is the ratio of two determinants, 
rather than the value of a given determinant, which is desired. The 
methods outlined above are useful in solving for determinantal ratios 
and, indeed, the forward solutions of the ten methods of solving simul- 
taneous equations are immediately obtained. It is hence improper to 
call these “non-determinantal solutions” since these are the solutions 
which improved determinantal methods indicate. 

Consider the equations 


Oy, Ly + Ag; Lo + Agi Lz = Myjy 
Ayo Ly + Age Le + Azo Xz = Ajo 
O13 Ly + Ass Lo + Msg Xz — Djs. 


If we solve for x; , the denominator is the determinant of the co- 
efficients while the numerator is the denominator with a,;; replaced by 
a;,;. It can be shown that the determinantal ratios become 


@ ja-12 





(a) in the method of division: x; = 
33.12 


(b) in the methods of single division: x, = a ; 
*12 
(c) in the methods of multiplication and subtraction: 2,=——— ; 
33-12 

and these agree with the results obtained in the study of the solution 
of equations. 

It is also possible to evaluate determinatal ratios when the de- 
terminants are not of the same order. In multiple correlation theory, 


for example, it is desired to evaluate the ratio of a determinant 





to one of its principal minors. As an illustration we desire to find = 
44 


If the method of single division, or any of its variations, is used, we 
have 


A __ My Age. Ag3.12 Ues-r28 
Aus A131 Me2.1 U33.47 





= Qg4.123- 
If the method of multiplication and subtraction, or any of its varia- 
tions, is used, we have 


A a Agar23 
Aus 4, Ags. Ags.12 
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Thus, from Table 3, 
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A 
- = .5903 
while from Table 5, 
A .30744 .30744 
So (1.0000) (.8400) (.6200) Fe 5208 ans 
In general 
A = Oxce-12-.. Ea 
Axx Fi 
A AKK-12-++ K-1 
a tee Ago.1 Agg.19°** AKA K--12--- K2 


Conclusion 


It is shown that the same methods can be used in evaluating de- 
terminants that are used in solving simultaneous equations. Espe- 
cially to be recommended are the “Abbreviated Doolittle” and “Com- 
pact” methods if the determinant is symmetric or almost symmetric. 
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A NOTE ON THE DISCOVERY OF A G FACTOR BY MEANS OF 
THURSTONE’S CENTROID METHOD OF ANALYSIS 


J. P. GUILFORD 
UNIVERSITY OF SOUTHERN CALIFORNIA 


A fictitious factor matrix including 16 tests and 3 factors, one 
of which was a g factor, was prescribed. From it two typical factor 
problems, including errors of sampling, were derived. Students in 
training, without awareness of the factor patterns, arrived at es- 
sentially correct solutions by the use of Thurstone’s centroid method 
with rotation of axes. Errors in the calculated factor matrix were 
very close in size to the sampling errors in the correlation coefficients. 
It is concluded that a g factor need not escape detection by Thur- 
stone’s procedures if the criteria of complete simple structure are 
not demanded. 


It is sometimes asserted that by virtue of its very nature Thur- 
stone’s centroid method of factor analysis cannot discover a g factor 
even when such a factor exists in a battery of tests. Thurstone’s com- 
plete criteria for a unique solution was originally embraced in what 
he called “simple structure.”* Complete simple structure calls for at 
least one zero loading in every row of the factor matrix and at least 
as many zeros in every column as there are factors, when the columns 
of the matrix correspond to the respective common factors and the 
rows to the respective tests. Since each factor, according to these re- 
quirements, must have at least one zero loading, it would follow that 
either there can be no g factor or there can be no unique solution. 

As a matter of practice, however, complete simple structure not 
always obtains and yet unique solutions of a kind are possible. As a 
general rule, the configuration of test vectors in the common-factor 
space is such that the reference axes (usually orthogonal) find natu- 
ral positions projecting through distinct clusters of those test vectors. 
The practical goal then becomes one of maximizing the number of 
zero factor loadings. When dealing with tests of abilities there is al- 
most invariably a positive manifold to aid as another criterion. When 
these two conditions are fulfilled, a maximal number of zero loadings 
and a positive manifold, the reference axes usually have psychological 
meaning. 

Even with these more liberalized criteria of a unique solution, 


* Thurstone, L. L. The vectors of mind. Chicago: University of Chicago 
Press, 1925, 156. 
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however, it might be thought that there is grave danger of inadver- 
tently missing a g factor. The mental set for zero loadings in every 
column of the factorial matrix must be recognized as a definite bias 
which might distort the solution. In this note is reported some evi- 
dence that a g factor will not necessarily escape Thurstone’s instru- 
ments of analysis. The brief study also brings out one or two other 
confirming sidelights with regard to the validity of Thurstone’s pro- 
cedures. 

A fictitious factor matrix (see Table 1) was set up with 3 com- 
mon factors and 16 tests. Factor I was a g factor and factors II and 
III were not. To make the problem more exacting, two low factor 
loadings of .2 and .3 and two others of .4 were introduced into column 
1 of the matrix. Ten zero loadings were introduced into the other two 
columns, since this is about the number that should be expected had 
there been complete simple structure in the matrix. 

From the factor matrix the corresponding correlation matrix 
was computed. In order to make the problem more true to life, sam- 
pling errors were introduced at random assuming a population of 200. 
This process of introducing errors was carried out two times, making 
two slightly different correlation matrices. The error increments 
ranged from —.13 to +.16 in onematrix and from —.14 to +.14 in 
the other. 

The two factor problems were presented to a class of four gradu- 
ate students who were gaining their first instruction in factor meth- 
ods. Two students solved each problem. They were told nothing con- 
cerning the source of the data, the number of common factors, or the 
fact that there was a g factor. From all they had learned previously 
in the course they should have been skeptical of the reality of a g fac- 
tor except under special conditions. They were told to extract the cen- 
troid factors, to apply various tests of when to cease extracting, and 
then to rotate the reference axes observing the usual practical cri- 
teria, a positive manifold and a maximal number of zeros. According 
to all the criteria of the correct number of factors, the rank of the 
correlation matrix was three. 

The procedure followed in rotation of axes was first the graphic 
method, with unextended vectors in planes, then later with extended 
vectors on the surface of a sphere. In all cases the first method gave 
the most valid results. In all cases there was no apparent escape from 
one g factor, in spite of the fact that the students tried to introduce 
zero loadings into every column. The results of the best solution are 
listed in Table 1. It is of considerable interest to note that the dis- 
crepancies between the original and the computed loadings are rather 
small for this solution. The range (see Table 1) is from —.17 to +.12. 
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The standard deviation of the discrepancies is .074, which is to be 
compared with the standard deviation of the original error incre- 
ments, which was .075. The magnitude of error in the rotated load- 
ings is no greater than that of the original correlation coefficients 
from which they came! This result should be heartening to those who, 
for lack of a rational method of finding standard errors of factor load- 
ings, are worried lest factor loadings are heavily contaminated with 
errors of sampling. 

In his recent volume on factor theory, Thomson gives several 
approximation tests of the extensiveness of the most extensive factor 
in a battery of tests.* A simple form of the test is to compare the larg- 
est latent root, approximated by means of the ratio S7i; ( where Sri: 
is the sum of all values in the correlation matrix, with the value of 1.00 
in each diagonal cell, and n is the number of test variables) with suc- 
cessive sums of the communalities in order of size. In one of the two 
factor problems, for example, the largest latent root was estimated as 
12.222. Summing the communalities, we find that the sum exceeds 
12.222 only after all 16 communalities are included, at which point 
the sum is 13,498. From this we conclude that a g factor is present. 

This simple and somewhat special problem does not of course 
settle the question with which we started. One cannot by any means 
conclude from it that Thurstone’s method will always find g factors 
when they are present. What it does show is that a g factor will not 
necessarily escape detection when his procedures are employed, and 
we believe that the problem is typical enough to enable us to predict 
that a g factor if present will usually be discovered, if we do not de- 
mand simple structure but adhere to the more practical criteria of a 
positive manifold and a maximal number of zeros when abilities are 
concerned. 


* Thomson, G. H. The factorial analysis of human ability. New York: 
Houghton Mifflin Company, 1989, Ch. XVII. 
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TABLE 1 
FACTOR MATRICES, ORIGINAL AND CALCULATED, WITH COMMUNALITIES 


Original Matrix Calculated Matrix 
Test k, k, &k, k’, k’, 





.03 
—.08 
73 
55 
—.02 
55 
52 
01 
17 
34 
58 
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TABLE 2 


CRITERIA OF SUFFICIENT EXTRACTION OF FACTORS 


Centroid Range °, Maximum =|p| =k 6 = .938 
Factor Loadings k2 (Tucker) 





83 479 
27 : 806 
24 : 681 
.08 ‘ 956 
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